{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 量子神经网络在自然语言处理中的应用\n",
    "\n",
    "[![下载Notebook](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_notebook.png)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/master/mindquantum/zh_cn/mindspore_qnn_for_nlp.ipynb)&emsp;[![下载样例代码](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_download_code.png)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/master/mindquantum/zh_cn/mindspore_qnn_for_nlp.py)&emsp;[![查看源文件](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/master/docs/mindquantum/docs/source_zh_cn/qnn_for_nlp.ipynb)\n",
    "\n",
    "## 概述\n",
    "\n",
    "在自然语言处理过程中，词嵌入（Word embedding）是其中的重要步骤，它是一个将高维度空间的词向量嵌入到一个维数更低的连续向量空间的过程。当给予神经网络的语料信息不断增加时，网络的训练过程将越来越困难。利用量子力学的态叠加和纠缠等特性，我们可以利用量子神经网络来处理这些经典语料信息，加入其训练过程，并提高收敛精度。下面，我们将简单地搭建一个量子经典混合神经网络来完成一个词嵌入任务。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "导入本教程所依赖模块"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import time\n",
    "import mindspore as ms\n",
    "import mindspore.ops as ops\n",
    "import mindspore.dataset as ds\n",
    "from mindspore import nn\n",
    "from mindquantum.framework import MQLayer\n",
    "from mindquantum.core.gates import RX, RY, X, H\n",
    "from mindquantum.core.circuit import Circuit, UN\n",
    "from mindquantum.core.operators import Hamiltonian, QubitOperator"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本教程实现的是一个[CBOW模型](https://blog.csdn.net/u010665216/article/details/78724856)，即利用某个词所处的环境来预测该词。例如对于“I love natural language processing”这句话，我们可以将其切分为5个词，\\[\"I\", \"love\", \"natural\", \"language\", \"processing”\\]，在所选窗口为2时，我们要处理的问题是利用\\[\"I\", \"love\", \"language\", \"processing\"\\]来预测出目标词汇\"natural\"。这里我们以窗口为2为例，搭建如下的量子神经网络，来完成词嵌入任务。\n",
    "\n",
    "![quantum word embedding](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/docs/mindquantum/docs/source_zh_cn/images/qcbow.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里，编码线路会将\"I\"、\"love\"、\"language\"和\"processing\"的编码信息编码到量子线路中，待训练的量子线路由四个Ansatz线路构成，最后我们在量子线路末端对量子比特做$\\text{Z}$基矢上的测量，具体所需测量的比特的个数由所需嵌入空间的维数确定。\n",
    "\n",
    "## 数据预处理\n",
    "\n",
    "我们对所需要处理的语句进行处理，生成关于该句子的词典，并根据窗口大小来生成样本点。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def GenerateWordDictAndSample(corpus, window=2):\n",
    "    all_words = corpus.split()\n",
    "    word_set = list(set(all_words))\n",
    "    word_set.sort()\n",
    "    word_dict = {w: i for i, w in enumerate(word_set)}\n",
    "    sampling = []\n",
    "    for index, _ in enumerate(all_words[window:-window]):\n",
    "        around = []\n",
    "        for i in range(index, index + 2*window + 1):\n",
    "            if i != index + window:\n",
    "                around.append(all_words[i])\n",
    "        sampling.append([around, all_words[index + window]])\n",
    "    return word_dict, sampling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'I': 0, 'language': 1, 'love': 2, 'natural': 3, 'processing': 4}\n",
      "word dict size:  5\n",
      "samples:  [[['I', 'love', 'language', 'processing'], 'natural']]\n",
      "number of samples:  1\n"
     ]
    }
   ],
   "source": [
    "word_dict, sample = GenerateWordDictAndSample(\"I love natural language processing\")\n",
    "print(word_dict)\n",
    "print('word dict size: ', len(word_dict))\n",
    "print('samples: ', sample)\n",
    "print('number of samples: ', len(sample))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "根据如上信息，我们得到该句子的词典大小为5，能够产生一个样本点。\n",
    "\n",
    "## 编码线路\n",
    "\n",
    "为了简单起见，我们使用的编码线路由$\\text{RX}$旋转门构成，结构如下。\n",
    "\n",
    "![encoder circuit](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/docs/mindquantum/docs/source_zh_cn/images/encoder.png)\n",
    "\n",
    "我们对每个量子门都作用一个$\\text{RX}$旋转门。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def GenerateEncoderCircuit(n_qubits, prefix=''):\n",
    "    if prefix and prefix[-1] != '_':\n",
    "        prefix += '_'\n",
    "    circ = Circuit()\n",
    "    for i in range(n_qubits):\n",
    "        circ += RX(prefix + str(i)).on(i)\n",
    "    return circ.as_encoder()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": "<div class=\"nb-html-output output_area\"><svg xmlns=\"http://www.w3.org/2000/svg\" width=\"156.8\" height=\"200\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n<rect x=\"0\" y=\"0\" width=\"156.8\" height=\"200\" fill=\"#ffffff\" />\n<text x=\"20.0\" y=\"40.0\" font-size=\"16px\" dominant-baseline=\"middle\" text-anchor=\"start\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#252b3a\" >\nq0:\n </text>\n<text x=\"20.0\" y=\"100.0\" font-size=\"16px\" dominant-baseline=\"middle\" text-anchor=\"start\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#252b3a\" >\nq1:\n </text>\n<text x=\"20.0\" y=\"160.0\" font-size=\"16px\" dominant-baseline=\"middle\" text-anchor=\"start\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#252b3a\" >\nq2:\n </text>\n<line x1=\"48.8\" x2=\"136.8\" y1=\"40.0\" y2=\"40.0\" stroke=\"#adb0b8\" stroke-width=\"1\" />\n<line x1=\"48.8\" x2=\"136.8\" y1=\"100.0\" y2=\"100.0\" stroke=\"#adb0b8\" stroke-width=\"1\" />\n<line x1=\"48.8\" x2=\"136.8\" y1=\"160.0\" y2=\"160.0\" stroke=\"#adb0b8\" stroke-width=\"1\" />\n\n<rect x=\"72.8\" y=\"20.0\" width=\"40.0\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#fac209\" fill-opacity=\"1\" />\n<text x=\"92.8\" y=\"36.0\" font-size=\"20px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\nRX\n </text>\n<text x=\"92.8\" y=\"52.0\" font-size=\"14.0px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\ne_0\n </text>\n\n\n<rect x=\"72.8\" y=\"80.0\" width=\"40.0\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#fac209\" fill-opacity=\"1\" />\n<text x=\"92.8\" y=\"96.0\" font-size=\"20px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\nRX\n </text>\n<text x=\"92.8\" y=\"112.0\" font-size=\"14.0px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\ne_1\n </text>\n\n\n<rect x=\"72.8\" y=\"140.0\" width=\"40.0\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#fac209\" fill-opacity=\"1\" />\n<text x=\"92.8\" y=\"156.0\" font-size=\"20px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\nRX\n </text>\n<text x=\"92.8\" y=\"172.0\" font-size=\"14.0px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\ne_2\n </text>\n\n</svg></div>",
      "text/plain": [
       "<mindquantum.io.display.circuit_svg_drawer.SVGCircuit at 0x7f2998704a60>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "GenerateEncoderCircuit(3, prefix='e').svg()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们通常用$\\left|0\\right>$和$\\left|1\\right>$来标记二能级量子比特的两个状态，由态叠加原理，量子比特还可以处于这两个状态的叠加态：\n",
    "\n",
    "$$\\left|\\psi\\right>=\\alpha\\left|0\\right>+\\beta\\left|1\\right>$$\n",
    "\n",
    "对于$n$比特的量子态，其将处于$2^n$维的希尔伯特空间中。对于上面由5个词构成的词典，我们只需要$\\lceil \\log_2 5 \\rceil=3$个量子比特即可完成编码，这也体现出量子计算的优越性。\n",
    "\n",
    "例如对于上面词典中的\"love\"，其对应的标签为2，2的二进制表示为`010`，我们只需将编码线路中的`e_0`、`e_1`和`e_2`分别设为$0$、$\\pi$和$0$即可。下面来验证一下。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Label is:  2\n",
      "Binary label is:  010\n",
      "Parameters of encoder is: \n",
      " [0.      3.14159 0.     ]\n",
      "Encoder circuit is: \n",
      " q0: ──RX(e_0)──\n",
      "\n",
      "q1: ──RX(e_1)──\n",
      "\n",
      "q2: ──RX(e_2)──\n",
      "Encoder parameter names are: \n",
      " ['e_0', 'e_1', 'e_2']\n",
      "Amplitude of quantum state is: \n",
      " [0. 0. 1. 0. 0. 0. 0. 0.]\n",
      "Label in quantum state is:  2\n"
     ]
    }
   ],
   "source": [
    "from mindquantum.simulator import Simulator\n",
    "\n",
    "n_qubits = 3 # number of qubits of this quantum circuit\n",
    "label = 2 # label need to encode\n",
    "label_bin = bin(label)[-1: 1: -1].ljust(n_qubits, '0') # binary form of label\n",
    "label_array = np.array([int(i)*np.pi for i in label_bin]).astype(np.float32) # parameter value of encoder\n",
    "encoder = GenerateEncoderCircuit(n_qubits, prefix='e') # encoder circuit\n",
    "encoder_params_names = encoder.params_name # parameter names of encoder\n",
    "\n",
    "print(\"Label is: \", label)\n",
    "print(\"Binary label is: \", label_bin)\n",
    "print(\"Parameters of encoder is: \\n\", np.round(label_array, 5))\n",
    "print(\"Encoder circuit is: \\n\", encoder)\n",
    "print(\"Encoder parameter names are: \\n\", encoder_params_names)\n",
    "\n",
    "# quantum state evolution operator\n",
    "state = encoder.get_qs(pr=dict(zip(encoder_params_names, label_array)))\n",
    "amp = np.round(np.abs(state)**2, 3)\n",
    "\n",
    "print(\"Amplitude of quantum state is: \\n\", amp)\n",
    "print(\"Label in quantum state is: \", np.argmax(amp))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过上面的验证，我们发现，对于标签为2的数据，最后得到量子态的振幅最大的位置也是2，因此得到的量子态正是对输入标签的编码。我们将对数据编码生成参数数值的过程总结成如下函数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def GenerateTrainData(sample, word_dict):\n",
    "    n_qubits = np.int(np.ceil(np.log2(1 + max(word_dict.values()))))\n",
    "    data_x = []\n",
    "    data_y = []\n",
    "    for around, center in sample:\n",
    "        data_x.append([])\n",
    "        for word in around:\n",
    "            label = word_dict[word]\n",
    "            label_bin = bin(label)[-1: 1: -1].ljust(n_qubits, '0')\n",
    "            label_array = [int(i)*np.pi for i in label_bin]\n",
    "            data_x[-1].extend(label_array)\n",
    "        data_y.append(word_dict[center])\n",
    "    return np.array(data_x).astype(np.float32), np.array(data_y).astype(np.int32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-7-4e49928cbb0a>:2: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.\n",
      "Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
      "  n_qubits = np.int(np.ceil(np.log2(1 + max(word_dict.values()))))\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(array([[0.       , 0.       , 0.       , 0.       , 3.1415927, 0.       ,\n",
       "         3.1415927, 0.       , 0.       , 0.       , 0.       , 3.1415927]],\n",
       "       dtype=float32),\n",
       " array([3], dtype=int32))"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "GenerateTrainData(sample, word_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "根据上面的结果，我们将4个输入的词编码的信息合并为一个更长向量，便于后续神经网络调用。\n",
    "\n",
    "## Ansatz线路\n",
    "\n",
    "Ansatz线路的选择多种多样，我们选择如下的量子线路作为Ansatz线路，它的一个单元由一层$\\text{RY}$门和一层$\\text{CNOT}$门构成，对此单元重复$p$次构成整个Ansatz线路。\n",
    "\n",
    "![ansatz circuit](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/docs/mindquantum/docs/source_zh_cn/images/ansatz.png)\n",
    "\n",
    "定义如下函数生成Ansatz线路。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def GenerateAnsatzCircuit(n_qubits, layers, prefix=''):\n",
    "    if prefix and prefix[-1] != '_':\n",
    "        prefix += '_'\n",
    "    circ = Circuit()\n",
    "    for l in range(layers):\n",
    "        for i in range(n_qubits):\n",
    "            circ += RY(prefix + str(l) + '_' + str(i)).on(i)\n",
    "        for i in range(l % 2, n_qubits, 2):\n",
    "            if i < n_qubits and i + 1 < n_qubits:\n",
    "                circ += X.on(i + 1, i)\n",
    "    return circ.as_ansatz()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": "<div class=\"nb-html-output output_area\"><svg xmlns=\"http://www.w3.org/2000/svg\" width=\"416.8\" height=\"320\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n<rect x=\"0\" y=\"0\" width=\"416.8\" height=\"320\" fill=\"#ffffff\" />\n<text x=\"20.0\" y=\"40.0\" font-size=\"16px\" dominant-baseline=\"middle\" text-anchor=\"start\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#252b3a\" >\nq0:\n </text>\n<text x=\"20.0\" y=\"100.0\" font-size=\"16px\" dominant-baseline=\"middle\" text-anchor=\"start\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#252b3a\" >\nq1:\n </text>\n<text x=\"20.0\" y=\"160.0\" font-size=\"16px\" dominant-baseline=\"middle\" text-anchor=\"start\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#252b3a\" >\nq2:\n </text>\n<text x=\"20.0\" y=\"220.0\" font-size=\"16px\" dominant-baseline=\"middle\" text-anchor=\"start\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#252b3a\" >\nq3:\n </text>\n<text x=\"20.0\" y=\"280.0\" font-size=\"16px\" dominant-baseline=\"middle\" text-anchor=\"start\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#252b3a\" >\nq4:\n </text>\n<line x1=\"48.8\" x2=\"396.8\" y1=\"40.0\" y2=\"40.0\" stroke=\"#adb0b8\" stroke-width=\"1\" />\n<line x1=\"48.8\" x2=\"396.8\" y1=\"100.0\" y2=\"100.0\" stroke=\"#adb0b8\" stroke-width=\"1\" />\n<line x1=\"48.8\" x2=\"396.8\" y1=\"160.0\" y2=\"160.0\" stroke=\"#adb0b8\" stroke-width=\"1\" />\n<line x1=\"48.8\" x2=\"396.8\" y1=\"220.0\" y2=\"220.0\" stroke=\"#adb0b8\" stroke-width=\"1\" />\n<line x1=\"48.8\" x2=\"396.8\" y1=\"280.0\" y2=\"280.0\" stroke=\"#adb0b8\" stroke-width=\"1\" />\n\n<rect x=\"72.8\" y=\"20.0\" width=\"80.0\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#fac209\" fill-opacity=\"1\" />\n<text x=\"112.8\" y=\"36.0\" font-size=\"20px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\nRY\n </text>\n<text x=\"112.8\" y=\"52.0\" font-size=\"14.0px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\na_0_0\n </text>\n\n\n<rect x=\"72.8\" y=\"80.0\" width=\"80.0\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#fac209\" fill-opacity=\"1\" />\n<text x=\"112.8\" y=\"96.0\" font-size=\"20px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\nRY\n </text>\n<text x=\"112.8\" y=\"112.0\" font-size=\"14.0px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\na_0_1\n </text>\n\n\n<rect x=\"72.8\" y=\"140.0\" width=\"80.0\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#fac209\" fill-opacity=\"1\" />\n<text x=\"112.8\" y=\"156.0\" font-size=\"20px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\nRY\n </text>\n<text x=\"112.8\" y=\"172.0\" font-size=\"14.0px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\na_0_2\n </text>\n\n\n<rect x=\"72.8\" y=\"200.0\" width=\"80.0\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#fac209\" fill-opacity=\"1\" />\n<text x=\"112.8\" y=\"216.0\" font-size=\"20px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\nRY\n </text>\n<text x=\"112.8\" y=\"232.0\" font-size=\"14.0px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\na_0_3\n </text>\n\n\n<rect x=\"72.8\" y=\"260.0\" width=\"80.0\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#fac209\" fill-opacity=\"1\" />\n<text x=\"112.8\" y=\"276.0\" font-size=\"20px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\nRY\n </text>\n<text x=\"112.8\" y=\"292.0\" font-size=\"14.0px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\na_0_4\n </text>\n\n<circle cx=\"192.8\" cy=\"40.0\" r=\"4\" fill=\"#16acff\" />\n<line x1=\"192.8\" x2=\"192.8\" y1=\"40.0\" y2=\"100.0\" stroke=\"#16acff\" stroke-width=\"3\" />\n<rect x=\"172.8\" y=\"80.0\" width=\"40\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#16acff\" fill-opacity=\"1\" />\n<line x1=\"178.8\" x2=\"206.8\" y1=\"100.0\" y2=\"100.0\" stroke=\"#ffffff\" stroke-width=\"4\" />\n<line x1=\"192.8\" x2=\"192.8\" y1=\"86.0\" y2=\"114.0\" stroke=\"#ffffff\" stroke-width=\"4\" />\n<circle cx=\"192.8\" cy=\"160.0\" r=\"4\" fill=\"#16acff\" />\n<line x1=\"192.8\" x2=\"192.8\" y1=\"160.0\" y2=\"220.0\" stroke=\"#16acff\" stroke-width=\"3\" />\n<rect x=\"172.8\" y=\"200.0\" width=\"40\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#16acff\" fill-opacity=\"1\" />\n<line x1=\"178.8\" x2=\"206.8\" y1=\"220.0\" y2=\"220.0\" stroke=\"#ffffff\" stroke-width=\"4\" />\n<line x1=\"192.8\" x2=\"192.8\" y1=\"206.0\" y2=\"234.0\" stroke=\"#ffffff\" stroke-width=\"4\" />\n\n<rect x=\"232.8\" y=\"20.0\" width=\"80.0\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#fac209\" fill-opacity=\"1\" />\n<text x=\"272.8\" y=\"36.0\" font-size=\"20px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\nRY\n </text>\n<text x=\"272.8\" y=\"52.0\" font-size=\"14.0px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\na_1_0\n </text>\n\n\n<rect x=\"232.8\" y=\"80.0\" width=\"80.0\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#fac209\" fill-opacity=\"1\" />\n<text x=\"272.8\" y=\"96.0\" font-size=\"20px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\nRY\n </text>\n<text x=\"272.8\" y=\"112.0\" font-size=\"14.0px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\na_1_1\n </text>\n\n\n<rect x=\"232.8\" y=\"140.0\" width=\"80.0\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#fac209\" fill-opacity=\"1\" />\n<text x=\"272.8\" y=\"156.0\" font-size=\"20px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\nRY\n </text>\n<text x=\"272.8\" y=\"172.0\" font-size=\"14.0px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\na_1_2\n </text>\n\n\n<rect x=\"232.8\" y=\"200.0\" width=\"80.0\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#fac209\" fill-opacity=\"1\" />\n<text x=\"272.8\" y=\"216.0\" font-size=\"20px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\nRY\n </text>\n<text x=\"272.8\" y=\"232.0\" font-size=\"14.0px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\na_1_3\n </text>\n\n\n<rect x=\"172.8\" y=\"260.0\" width=\"80.0\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#fac209\" fill-opacity=\"1\" />\n<text x=\"212.8\" y=\"276.0\" font-size=\"20px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\nRY\n </text>\n<text x=\"212.8\" y=\"292.0\" font-size=\"14.0px\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-family=\"Arial\" font-weight=\"normal\" fill=\"#ffffff\" >\na_1_4\n </text>\n\n<circle cx=\"352.8\" cy=\"100.0\" r=\"4\" fill=\"#16acff\" />\n<line x1=\"352.8\" x2=\"352.8\" y1=\"100.0\" y2=\"160.0\" stroke=\"#16acff\" stroke-width=\"3\" />\n<rect x=\"332.8\" y=\"140.0\" width=\"40\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#16acff\" fill-opacity=\"1\" />\n<line x1=\"338.8\" x2=\"366.8\" y1=\"160.0\" y2=\"160.0\" stroke=\"#ffffff\" stroke-width=\"4\" />\n<line x1=\"352.8\" x2=\"352.8\" y1=\"146.0\" y2=\"174.0\" stroke=\"#ffffff\" stroke-width=\"4\" />\n<circle cx=\"352.8\" cy=\"220.0\" r=\"4\" fill=\"#16acff\" />\n<line x1=\"352.8\" x2=\"352.8\" y1=\"220.0\" y2=\"280.0\" stroke=\"#16acff\" stroke-width=\"3\" />\n<rect x=\"332.8\" y=\"260.0\" width=\"40\" height=\"40\" rx=\"4\" ry=\"4\" stroke=\"#ffffff\" stroke-width=\"0\" fill=\"#16acff\" fill-opacity=\"1\" />\n<line x1=\"338.8\" x2=\"366.8\" y1=\"280.0\" y2=\"280.0\" stroke=\"#ffffff\" stroke-width=\"4\" />\n<line x1=\"352.8\" x2=\"352.8\" y1=\"266.0\" y2=\"294.0\" stroke=\"#ffffff\" stroke-width=\"4\" />\n</svg></div>",
      "text/plain": [
       "<mindquantum.io.display.circuit_svg_drawer.SVGCircuit at 0x7f29986ff100>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "GenerateAnsatzCircuit(5, 2, 'a').svg()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 测量\n",
    "\n",
    "我们把对不同比特位上的测量结果作为降维后的数据。具体过程与比特编码类似，例如当我们想将词向量降维为五维向量时，对于第三维的数据可以如下产生：\n",
    "\n",
    "- 3对应的二进制为`00011`。\n",
    "- 测量量子线路末态对$Z_0Z_1$哈密顿量的期望值。\n",
    "\n",
    "下面函数将给出产生各个维度上数据所需的哈密顿量（hams），其中`n_qubits`表示线路的比特数，`dims`表示词嵌入的维度："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def GenerateEmbeddingHamiltonian(dims, n_qubits):\n",
    "    hams = []\n",
    "    for i in range(dims):\n",
    "        s = ''\n",
    "        for j, k in enumerate(bin(i + 1)[-1:1:-1]):\n",
    "            if k == '1':\n",
    "                s = s + 'Z' + str(j) + ' '\n",
    "        hams.append(Hamiltonian(QubitOperator(s)))\n",
    "    return hams"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1 [Z0] , 1 [Z1] , 1 [Z0 Z1] , 1 [Z2] , 1 [Z0 Z2] ]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "GenerateEmbeddingHamiltonian(5, 5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 量子版词向量嵌入层\n",
    "\n",
    "量子版词向量嵌入层结合前面的编码量子线路和待训练量子线路，以及测量哈密顿量，将`num_embedding`个词嵌入为`embedding_dim`维的词向量。这里我们还在量子线路的最开始加上了Hadamard门，将初态制备为均匀叠加态，用以提高量子神经网络的表达能力。\n",
    "\n",
    "下面，我们定义量子嵌入层，它将返回一个量子线路模拟算子。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def QEmbedding(num_embedding, embedding_dim, window, layers, n_threads):\n",
    "    n_qubits = int(np.ceil(np.log2(num_embedding)))\n",
    "    hams = GenerateEmbeddingHamiltonian(embedding_dim, n_qubits)\n",
    "    circ = Circuit()\n",
    "    circ = UN(H, n_qubits)\n",
    "    encoder_param_name = []\n",
    "    ansatz_param_name = []\n",
    "    for w in range(2 * window):\n",
    "        encoder = GenerateEncoderCircuit(n_qubits, 'Encoder_' + str(w))\n",
    "        ansatz = GenerateAnsatzCircuit(n_qubits, layers, 'Ansatz_' + str(w))\n",
    "        encoder.no_grad()\n",
    "        circ += encoder\n",
    "        circ += ansatz\n",
    "        encoder_param_name.extend(encoder.params_name)\n",
    "        ansatz_param_name.extend(ansatz.params_name)\n",
    "    grad_ops = Simulator('mqvector', circ.n_qubits).get_expectation_with_grad(hams,\n",
    "                                                                              circ,\n",
    "                                                                              parallel_worker=n_threads)\n",
    "    return MQLayer(grad_ops)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "整个训练模型跟经典网络类似，由一个嵌入层和两个全连通层构成，然而此处的嵌入层是由量子神经网络构成。下面定义量子神经网络CBOW。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CBOW(nn.Cell):\n",
    "    def __init__(self, num_embedding, embedding_dim, window, layers, n_threads,\n",
    "                 hidden_dim):\n",
    "        super(CBOW, self).__init__()\n",
    "        self.embedding = QEmbedding(num_embedding, embedding_dim, window,\n",
    "                                    layers, n_threads)\n",
    "        self.dense1 = nn.Dense(embedding_dim, hidden_dim)\n",
    "        self.dense2 = nn.Dense(hidden_dim, num_embedding)\n",
    "        self.relu = ops.ReLU()\n",
    "\n",
    "    def construct(self, x):\n",
    "        embed = self.embedding(x)\n",
    "        out = self.dense1(embed)\n",
    "        out = self.relu(out)\n",
    "        out = self.dense2(out)\n",
    "        return out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们对一个稍长的句子来进行训练。首先定义`LossMonitorWithCollection`用于监督收敛过程，并搜集收敛过程的损失。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "class LossMonitorWithCollection(ms.train.callback.LossMonitor):\n",
    "    def __init__(self, per_print_times=1):\n",
    "        super(LossMonitorWithCollection, self).__init__(per_print_times)\n",
    "        self.loss = []\n",
    "\n",
    "    def begin(self, run_context):\n",
    "        self.begin_time = time.time()\n",
    "\n",
    "    def end(self, run_context):\n",
    "        self.end_time = time.time()\n",
    "        print('Total time used: {}'.format(self.end_time - self.begin_time))\n",
    "\n",
    "    def epoch_begin(self, run_context):\n",
    "        self.epoch_begin_time = time.time()\n",
    "\n",
    "    def epoch_end(self, run_context):\n",
    "        cb_params = run_context.original_args()\n",
    "        self.epoch_end_time = time.time()\n",
    "        if self._per_print_times != 0 and cb_params.cur_step_num % self._per_print_times == 0:\n",
    "            print('')\n",
    "\n",
    "    def step_end(self, run_context):\n",
    "        cb_params = run_context.original_args()\n",
    "        loss = cb_params.net_outputs\n",
    "\n",
    "        if isinstance(loss, (tuple, list)):\n",
    "            if isinstance(loss[0], ms.Tensor) and isinstance(loss[0].asnumpy(), np.ndarray):\n",
    "                loss = loss[0]\n",
    "\n",
    "        if isinstance(loss, ms.Tensor) and isinstance(loss.asnumpy(), np.ndarray):\n",
    "            loss = np.mean(loss.asnumpy())\n",
    "\n",
    "        cur_step_in_epoch = (cb_params.cur_step_num - 1) % cb_params.batch_num + 1\n",
    "\n",
    "        if isinstance(loss, float) and (np.isnan(loss) or np.isinf(loss)):\n",
    "            raise ValueError(\"epoch: {} step: {}. Invalid loss, terminating training.\".format(\n",
    "                cb_params.cur_epoch_num, cur_step_in_epoch))\n",
    "        self.loss.append(loss)\n",
    "        if self._per_print_times != 0 and cb_params.cur_step_num % self._per_print_times == 0:\n",
    "            print(\"\\repoch: %+3s step: %+3s time: %5.5s, loss is %5.5s\" % (cb_params.cur_epoch_num, cur_step_in_epoch, time.time() - self.epoch_begin_time, loss), flush=True, end='')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，利用量子版本的`CBOW`来对一个长句进行词嵌入。运行之前请在终端运行`export OMP_NUM_THREADS=4`，将量子模拟器的线程数设置为4个，当所需模拟的量子系统比特数较多时，可设置更多的线程数来提高模拟效率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-7-4e49928cbb0a>:2: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.\n",
      "Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
      "  n_qubits = np.int(np.ceil(np.log2(1 + max(word_dict.values()))))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch:  25 step:  20 time: 0.351, loss is 3.154\n",
      "epoch:  50 step:  20 time: 0.362, loss is 3.023\n",
      "epoch:  75 step:  20 time: 0.353, loss is 2.948\n",
      "epoch: 100 step:  20 time: 0.389, loss is 2.299\n",
      "epoch: 125 step:  20 time: 0.392, loss is 0.810\n",
      "epoch: 150 step:  20 time: 0.389, loss is 0.464\n",
      "epoch: 175 step:  20 time: 0.384, loss is 0.306\n",
      "epoch: 200 step:  20 time: 0.383, loss is 0.217\n",
      "epoch: 225 step:  20 time: 0.387, loss is 0.168\n",
      "epoch: 250 step:  20 time: 0.382, loss is 0.143\n",
      "epoch: 275 step:  20 time: 0.389, loss is 0.130\n",
      "epoch: 300 step:  20 time: 0.386, loss is 0.122\n",
      "epoch: 325 step:  20 time: 0.408, loss is 0.117\n",
      "epoch: 350 step:  20 time: 0.492, loss is 0.102\n",
      "Total time used: 138.5629165172577\n"
     ]
    }
   ],
   "source": [
    "import mindspore as ms\n",
    "ms.set_context(mode=ms.PYNATIVE_MODE, device_target=\"CPU\")\n",
    "corpus = \"\"\"We are about to study the idea of a computational process.\n",
    "Computational processes are abstract beings that inhabit computers.\n",
    "As they evolve, processes manipulate other abstract things called data.\n",
    "The evolution of a process is directed by a pattern of rules\n",
    "called a program. People create programs to direct processes. In effect,\n",
    "we conjure the spirits of the computer with our spells.\"\"\"\n",
    "\n",
    "ms.set_seed(42)\n",
    "window_size = 2\n",
    "embedding_dim = 10\n",
    "hidden_dim = 128\n",
    "word_dict, sample = GenerateWordDictAndSample(corpus, window=window_size)\n",
    "train_x, train_y = GenerateTrainData(sample, word_dict)\n",
    "\n",
    "train_loader = ds.NumpySlicesDataset({\n",
    "    \"around\": train_x,\n",
    "    \"center\": train_y\n",
    "}, shuffle=False).batch(3)\n",
    "net = CBOW(len(word_dict), embedding_dim, window_size, 3, 4, hidden_dim)\n",
    "net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')\n",
    "net_opt = nn.Momentum(net.trainable_params(), 0.01, 0.9)\n",
    "loss_monitor = LossMonitorWithCollection(500)\n",
    "model = ms.Model(net, net_loss, net_opt)\n",
    "model.train(350, train_loader, callbacks=[loss_monitor], dataset_sink_mode=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "打印收敛过程中的损失函数值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAxTUlEQVR4nO2de3gc9XX3P2d3JdmyZVu+4AtGNgbHMbca28FAUkoupEDTcEnCxXlD0lwMhPQtTfMUEnhJ6oaGpG2e0IY3xrk0IcWEBGNCeEkDSSmQYIQtc7GNYzDGMrLxXTKyZeu25/1jZ8RovXfN7Mxqz+d59Gh3Znb2SFrNd87ld46oKoZhGEb1EgvbAMMwDCNcTAgMwzCqHBMCwzCMKseEwDAMo8oxITAMw6hyEmEbUCwTJ07UmTNnhm2GYRhGRdHS0rJPVSdl2ldxQjBz5kzWrl0bthmGYRgVhYi0ZttnoSHDMIwqx4TAMAyjyjEhMAzDqHJMCAzDMKocEwLDMIwqx4TAMAyjyqm48tHhxIrm7dz1xKvsO9RD0ukCm1RFFWICSU19xQUQBrYrkEymzhGPpZ4Xug+gvibO5LEj+fS7T2Txoqay/syGYUQPqbQ21AsXLtQoryNY0bydH/1+K2++dZSjPf1IlotzfzJMK98mBkgsv7CIwKjaOHW1ccaNrDURMYwKQ0RaVHVhxn0mBKWTfkff219Zv8uhIs73ETUxRo1ImEAYRoQxIfCROx7dxD2rt9HVG5Fb+ggSFxhVl2DxWU3cfPHcsM0xDAMTgiHj3vm/efAoyYB/XXEnHON3jkCAMByWGDBmZIKJDSPMWzCMEAlVCEQkDqwFdqjqh9L21QH3AAuA/cCVqrot1/nKJQRurL+t4whHh3D3717EM12ca+MxTpw4ihPG1zOpoY7L509nwYxGX+zPREtrOyvXtbGvs5vNuzrZ0dEF5BcWP/MZU8bU8f65kwP/WQ3DGEzYQvBFYCEwJoMQfB44Q1WvE5GrgMtU9cpc5wtaCFpa27nhP1vY1dld9GtjAALHNQy/i50rIlt2d7J132E6j/bR05dkKJ+eS+dN4ztXnembjYZhZCc0IRCR6cBPgNuBL2YQgt8AX1PV1SKSAHYBkzSHUUEJwYrm7Xz78c3sO9RT8Gtq48JJk0Yzf0bjsLroF0O6QBw80ltU0jwRg/lNjdx00dyq/P0ZRrnIJQRBryP4DvD3QEOW/ccDbwCoap+IHAQmAPu8B4nIEmAJQFOTvzHmYgWgNi7MO2GcXbgcFsxoPOb30NLazq2r1vPK7s68eYm+JDy3rZ2PfO8Zjmuo5cYPzLE8gmGUmcA8AhH5EHCxqn5eRM4HvpTBI9gAXKiqbc7z14BFqrov/XwufnkExYaA7CJVGi2t7Sx78jWe2bKPwz39Bb1mwqgall/zLhNaw/CRUEJDIvIN4BNAHzACGAM8qKr/y3NMWUND7kXpudf3c/BIX97j62tivGf2JK79s5PsouQDxXpfp0xt4B8vPd1+94bhA6GXj+bwCG4ATvckiy9X1StynatUIbjj0U0se2prwcfXxYXuiCwQE6AuEeOM6WOHRUiqpbWdO369iXWt7QWVtJ4103IIhjFUIiUEIrIUWKuqD4vICOCnwJnAAeAqVc15tS5FCFY0b+crq9aXZHs1EBeIxYSYCGNGJDhx4igADhzuYfyoWsbV1w4c63eZazEL9EwQDKN0QhcCPylFCD7xw2aefjVr2sEogSlj6jhj+rhB2zq6eujuS3Llu5qKzqW0tLbzuZ+s4UBXb95jP3jKZAvXGUaRVL0QmEdQfuZOaeCE8fVAcV7Eiubt3P7/Xs6bWI4JfP3S0y15bxgFUvVCADD3//yaI9YfKDQEmNY4klOnjinobv7Gnz3PQy/szHteCxcZRmGYEFC6VxAXmDp2BJ9/7+zA7z69LSA6unoGVvD29icD73FUTgS4oIDwjrseYdOuzrzntHCRYeTGhMDhou88VdBFJRMxgYUzGhlXX0tHV8+gRGo5+gT5hVtC+/z2dtq7ekKdixAT+MDc/BfwQiu+LFxkGNkxIXBoaU2tYA0CVyggc7VNunh4KWVfvteUmrR1cZvu7T3cTVd3f6CzFkTg2j+dlbNltVtyumZbe+5zAbdfZmJgGOmYEHhYcs9aHnt5t48WRRtv0hZKExavx1NM+4hiKSTev6J5O7esWp+32Z3lDgxjMCYEHgr1CuoSMU5oHElNPMbm3Z3DKkZfKumi4grE5l2dLP3VRo72DT3OFI8J/3jJaTnv6Fc0b+f/PLS+ICGy3IFhpDAhSKOYXMGl86bxiXNmDiRxYfDdM8Da1vaqFQq3Guj4sSMG8icbd75VcF+hbOfMl0x2cx2PF+DdxWPw82vPNTEwqhoTgjRKzRXEYzBn8rH9b9KrfcLOEbxxoKvkpPhQcXMlnUf7hmxDIcnfFc3buXXVevL5IpNG17LsEwtNDIyqxYQgAx/49pNs2XPIB4uihwCx2OCpYyKpFhI1MSFJqp32qBE1g+7kMwnLUEVlSkNdSUN+vFx3Xv5EcqHeQb5zGcZwxYQgA7bauHASzui1hhFxZh/XMCAca7a1FzShbO6UBtoP9wxJEAqZZlZomal1NTWqEROCLLzr64+zt4iJZEYKcb8LjEjEQIQYcChPXmDcyAS9SeVwd2n5g0Iu4Hc8uom7n9pakECZd2BUE7mEIFZuY6KE22UzGw118TJZUlmo85VU6OpN0tXTn1cEADqO9HG4u7/kD93Lb3byke89wxXLnqGlNfN6gpsvnsvtl51OXDLuHsSyp7Zy48+eL9Eawxg+VLUQzJ6cbYJmis7ufq47bxYrrz+XC06ZTG0hVxcjL0MtMnVHW97x6KaM+xcvauLn16X+ZpLnT/bQCztNDIyqp6pDQy2t7Xx02TPk+xXkqkX3Vgy5BL3qN5MNy558jdf3HqImHuOto7109yfp7U9ytCc57HoVecm3cKzQBWiF5CAMo5KxHEEOCl1pXGhfnEolXdC8Te+SqvQnNbJiku9vU2gS2eZSG8OZsGYWjwCeAuqABPCAqn417ZhPAf8M7HA2fVdVf5DrvH4LQUtrOx/73jMFhysEOOm40Xz63SdW3QXD27Auil1RcwlCMUlk8w6M4UhYQiDAKFU9JCI1wO+Bv1HVZz3HfApYqKpfKPS8fgsBwC2r1nNv8/aiXzd3SgPzZzRWTOfRIHCbwb30RkdkZjxnW4i2onk7tz60viDxOm/2RO75zKKALDSM8hN6aEhE6kkJwfWq2uzZ/ikiIAQtre1cefczZGuVI5DzTjImqRXHvf1JZk0aPWzDR/kotENoOcjWhdTEwKhWQhMCEYkDLcDJwF2qelPa/k8B3wD2Aq8Af6uqb2Q4zxJgCUBTU9OC1tZW323N5xWcNbOx4AVUArxrZuNAojiIBHGUcfMNj2/cFfo6jUyJ/mKa1lmYyBguRMEjGAesAv5aVTd4tk8ADqlqt4hcC1ypqu/Lda4gPAJIXbyuXr6anixXhxnj6/n2lfOGfMdbbFvoSvYwWlrbuXL5avpCDhkJcG3a4rFi2lLYSmRjOBC6EDhG3AZ0qeq/ZNkfBw6o6thc5wlKCCC/VzBzQj3/esU8AFaua2PL7s6ydB7N5GGU0qQuDK/EveD+9uXdBXlTQZKpKqjQqjERuN2mnxkVTFjJ4klAr6p2iMhI4DHgm6r6iOeYqar6pvP4MuAmVT0713mDFIJUrmA1fXmu7N7adTcMsmV3Jzs6jrCz42joF7x85PNKghi9GaX8gfcOv6W1nSvuXk1/gWpu8w2MSiUsITgD+AkQJ7WC+eequlRElgJrVfVhEfkG8GGgDzhAKpn8x1znDVIIYOjx40wLzMJsC10qmUZvzp7cMGSBKLRtdNB4k8nF/M3B5hsYlUkkQkN+EbQQQHHlpPU1Md4ze1Leu8RMAgG5QzlR9DC8AlFqqKmltZ2bVr4UiTbgrncHpEJYm3bnXWkOqVDdL647N2DrDMM/TAiKpKW1nSuWPVP0TN5Jo2s5s6nR19BBKS0sgpgrkAs31FRMSCkqiWSXs2Y2MntyAw11CZY/vbWgvI+FiYxKwoSgBNKTiGfNbESh4Bh3bVyor40PDH/xI6wyVArxSmBoozddj6GQnzdKiWQXEfjTkyfy+y37CvodFDJj2TCigAlBCaTfsbr/8HOmNHDDf7aUPGSlJp6aFDayJkYiNfFloDkcMGi7d188JswYX1+WJG+m0Zs18Ribd3cWJRCFjJp03y8qiWSX82ZP5OlX9xW8biTT4jXDiBImBCWSnivwXtjueHQT9za30lnikJUgGV0bp9e5YmcTlkL2ud4McEy4qZhQkxt2ySdSUZsaN25kDR1Hegs+3gbdGFHGhKBEMpWTpt/lehux7auSaWfjRiYYNaKGMXUJ9h7qpqunv6AJZYV0cC20U2hUMTEwoooJwRDI1JsmW8jDDXFs3HGQfoWamBQ0uavayBdXr3Qx+CcLExkRxIRgiBQjBul4F5y5/f2BonIEXd399EakusZPcrVuiMp6g1KxiiIjapgQ+EAmMcjUwyYoCq342dFxhEM9fQXnAbLtK6c3k62dd0trO1+8/wVaD3SVxQ6/sYoiI0qYEPhEthbGwzUunF49tKPjyMAIzINdfb6/n9vOuzYRG7RQ7ZofNvPUq/t8f79yUKjnaBhBY0LgI9lm4FZbKMArEkEtVvN6Co9v3FWxeQMrLzWigAmBz2RLZg73uca58OZC8i1Ii8egv4jgv/t7ra+N89ALO4dubEhYEtkIExOCAMg1A1cErv3T4RkuKgRXFJ5vbc/qKQgwpj5RVIiplNdEDRMDIyxMCAIiX9dKm2lc3ND4amG45pSMaGNCECDugrLfbdqdNRzi9t8Jqtd/1Cnkd5RvLvRwI9OQHMMIEhOCMlBMv5z0iWPVIg5RbDIXNjMn1POOyQ1V8xkwwiOswTQjgKeAOiABPKCqX007pg64B1gA7Cc1s3hbrvNGVQhcSg2FCDCtcSTHjx0x7GYXp5OtDNclk3cgQNP4el/WFIysiXGkN3pL1dzy2d7+5LD6exvRICwhEGCUqh4SkRrg98DfqOqznmM+D5yhqteJyFXAZap6Za7zRl0IoLgKmmLIN7u4ku4qB7yDAgfBwNsL+F7bdzhnmClIBHjnlIayTJxL/3tD9XiPhv+EHhoSkXpSQnC9qjZ7tv8G+JqqrhaRBLALmKQ5jKoEIfDiFYU129oDD4mkj5iM+p1lsWMi3SHyc6Y0+BJmmlvCRd1P76QUst0QRP1vbYRLaEIgInGgBTgZuEtVb0rbvwG4UFXbnOevAYtUdV/acUuAJQBNTU0LWltbA7M5SLJNGwtaILwXjijeUebyDupr43SltboQ4AJnAR8w5FkG86aP5cW2g8Mib5HPawTzKqqVKHgE44BVwF+r6gbP9oKEwEuleQSF4F4IX997qCyzi6O6ziHbqu1seBfwAdy6an3JIZsxI+K8c8oYnitRUPysemqoi3Oouz9QYfLmI4oZeVrKmFQvmUSopbWdZ7fu5+xZEwBYua4NARMrnwldCBwjbgO6VPVfPNuGfWjIL/LNLobiR0y6g9uj9M+2onk7tzy0PmPeINvF1tvcbagtrOdOaaD9cE/RE+iOdxL9pQpJOufNnsj08fUDf+9yeI7lwltO/caBLv64uxNViJH6+7o/Y7YCCsicHzt12lie2Lwn4w2VSzV7Q2EliycBvaraISIjgceAb6rqI55jbgBO9ySLL1fVK3Kdt1qFoBAyNYnL50lEsS1GsXkDGNzPJ33edClMHzeCto6jRb0mHhM+954T+fWGXb7kD2ZOqOdfr5g38HfJ1YHWb69xOJMpCQ/DXyTCEoIzgJ8AcVJi/3NVXSoiS4G1qvqwU2L6U+BM4ABwlarmvJ0zISiOQpvDRU0Q8lUVZSwx9SSSr7h7Nf1DLCuqiUGxVaZu/uJob78vHVMFeOD6cwv6m+TzGsfV19LR1eNrJdtwI1OOpbsvOagbbqUSidCQX5gQDI186xyi1kM/15qDSaNr2Zs2HtS9EM+aOIrvP721KK/CT0Tgkj+Zxo6OI0NKZAOMqo1zy1+c4tvfxFvJVmy8v5QcQbHiE9VV5nOnNHDC+PqB55UmEiYExiDytXyIWtvkXKGimJDxZ3DDNGGuOYBUX6ELTp3iy4CdXBPdoo6bEG6sr2XDzoODch/dfUnOmTWBt7r7BpLEQNYCCvd13vxYy/YOkklFgKlZFmYGmWNxRSLK4SUTAiMjuQTBDbNERQxaWtu5aeVLbNlz6Jh9Extq2d/Zk3E18rXnzaKzu497m7eXxc5MuLMq7vztK76Ei6pt9kUheCuPsv1ecuVY/BSJbAOWwsaEwMhJtots1KZrtbS2c+Xy1fRlcA3E+Z7p0zx3SgOv7j2U8XVzpzQMVK0EiZuDmTVxFA+sa2NfWkirWMo5JrUayCYSfgxdikp1ngmBkZdsF9koikGuJPKMLCt+3bu09H/qKWPquHTe8WXLJ/gtCFG5yAxn/BAJt2R29uSG0EJHJgRGQWS7yCZiwv3XnhOpi02uvMG86WN5oe3gMduztYZw8wlvdffx1OY9RZeNlkIiLty/5Bw27+rkm/+1iYNHSh+2411pHaW/UTWQXrJdaFI8jFklJgRGUaRX6ghw9aIm/umy00O1Kx1XuB5PWzMgwJ/OnsjTr+4rOO7rXkzPn3Mctz28IWMYyW9OPm403/zIGSyY0cg1P2z2JX9gHkK4FDKdz0s5PQUTAqNo0u+4o1ZW6iXTamK3HhxgTWt7wTmAclcbee/mN+/q5K4nXmWHDx6JTUELn0IGMnkR4KTjRvPpd58YyP+ZCYFREresWj+o2iZq+QIv2dZHxGPCX54xlV+9uLPgHEBc4OfXnQuQ0eMIAr8b6QE01id418wJFjIKmWK9BAimHNWEwCiJltZ2rrx7NX2e25ko5gtcsi0+G+QdFFgm6A3bFDJ9ri4Ro7vPn2E3l86bxneuOpOW1nZf1h9AavHdmU2NJgohs6J5O/ev2U5PX5LNuzsL8hT8Ch+ZEBglkylf8KU/n8MN7z05VLuykSuJnIgLn333idz99NaCQkXp4bAbf/Y8D72wM+drRtXFOdzdn/OYQqhLxPjqX57K4kVNrGjeztJfbeSoT0JjeYRoUMqsErfIoJS/nQmBMSTSY/D/FKFVx5nIVWJ68nGj+cA7jyu4XDS9IqfQpK5fglAbF+adMI6bLprLT1dvyytExWBeQnQoZub54hILN0wIjCFx1xNb+NfHNpPU6FYQZSLbfAPvFLdCy/1KbXU9MhHjiE938nGB8aNqqa9N+D4drb42zmnTxpinEDKFlKN+fFETt5sQmBCUm5bWdq5evpoe5xY6yhVE6eSab1CsIHhX8xZzBzdjfD07Dx6h18eS1CAbs/nhKWRbhAWlN67LNfTGm1R131uAhroEq7fupy4RGzShzd0+ecyIgn7OQlpYBIG38kgVahIx7vvc2RYaMiEIh/QKoignjdPJN9/A7RS6YedbGXsZpeONsa9o3s5XVq3P+5qodtTMhwA1cWHsyBpOnDiqoAt3WEN0BHjnlIaCk7De1+Ua79nR1VNQU7t84jZr0mjOn3PcQNM9V7yAgvokDVWITAiMIZNeQVRJISIorKZ77pQGXtnTSX8BkRyvV1SoGBhGJtybhLjAAmdyWxCzpk0IDF9Iv7OuHYKbGhaFtOCekKWbaaZjvYvBbl21Hn+yAYaRmaH8z+USgsSQLcv+picA9wCTSQneclW9M+2Y84FfAq87mx5U1aVB2WQMjcWLmti48yArmrejQH9/kme37q8oIVgwo5HvX7Mwa4xfgX2dPQjZ21t7j33s5d38dtNuPjB3Ml+/7HSe2LyH3768u+CwyOjaOId6Cqsu8qsSqZxkGgvpZ44gWxhKSIX83AKHdzoLtNwwT7/Hs62kW+HevmD+5wITAqAP+DtVXSciDUCLiDyuqi+nHfe0qn4oQDsMH7l8/nRWrmujpzeJiNCYIXFXCSyY0cgvrjs364pkVxAAxtYnONiVvSlcUt8WhK9fejrvnXNc1qlq6RQqAgCHu/uZO6WBt472Vsx84ngMXtndycSGEYG1TkhPTOeLvXvj7UDe8Z7uOU+dNpYnNu/JOCwnn7jt6DjCmwePopoqUnjH5Ab+uKuz6L9hTSI2YLeflC00JCK/BL6rqo97tp0PfKkYIbDQUPisaN7Obb/cQFKV2kSMez9bWeGhdIrtCZMLd6APkDNBPVSKCWFFjZikxm/W1cYBqEvEj0m+QuWNgsxHesLX/dzteevowIQ27+S2YZcjEJGZwFPAaar6lmf7+cBKoA3YSUoUNmZ4/RJgCUBTU9OC1tbWwG02slOp6wryMbAQrYjQTjY+6HQyXfV825B7BuVj3MgEB4/0VZwgFMPMCfW8Y3IDcOxFMsrjIaNEqEIgIqOBJ4HbVfXBtH1jgKSqHhKRi4E7VXV2rvOZRxA+6esKKjFpnItsPYuKxU0m19fG+eULOwO/UE9sSN05VqKXMFS8a0IyhWhMLEIUAhGpAR4BfqOq3y7g+G3AQlXNuobfhCAa3LJq/UDSeDh5BS7ePjCFrj7OhrtOoZgOqEOlvjZOV4H5h7iTVB3u4iHAtLQ1AG7sv72rp+yLxcpNKEIgIgL8BDigqjdmOWYKsFtVVUTOAh4AZmgOo0wIosFw9wq8+JVDmDG+nr5ksqyJ3oa6VCy+0HLYeAx86ohRkbi/L3g7dxHmeEk/CUsI3gM8DayHgfLqrwBNAKq6TES+AFxPqsLoCPBFVX0m13lNCKKD1yuIC3zxg9HtSuoHfuYQxo1M0DGE8ZSlvqcKOSugjOyki8SYugS1iRjnzJpAw8iayHsUoSeL/cSEIDq4XkFvv1ITF+4rsT1upeFn2Cgs4kLZwlTVRENdnDH1tYypS9Dbn8zaGwkGl7l6S1hzlaIOJdcRyoIyo0oQZ0mOSNiWlI0FMxoH/hHTu0WG0WOnFLwikKjycJCfdHb309l9hB3uhr2Hcx7v7d91DFle+4uWNt/DsCYERsk8u3U/ff3Jil1l7AdeUYBjheG5gEtH/cBEoLIIYnWxCYFRMmfPmkBtIlbxq4z9JJswPN/aTuv+w3T12lXXGBqxmPi+utiEwCiZBTMaue1Dpw6sMl76yEbmTGmoOq8gF5mEwY/B9Eb1cuKEet//x2K+ns2oOtq7ekiqktS3XVYjO26Po5XXn8viRU3MndIQtklGicRILWQrNzVx/y/bBXkEIjIKOKKqSRF5B/BO4Neq2uu7RUZFYeGh0khPOJuXUHkkIZRVeJt3d9LS2u6rV1CotDwFjBCR44HHgE8AP/bNCqNiccNDsZgMhIdaWu2CVgzpXsKUhrqwTRqWiPNV6ajiu+ddaI5AVLVLRD4D/F9V/ZaIvOCrJUbFkik8ZHmC4hnwEi47fWDx2vPb29l3qCds04YFlVDWWwhBtKIuWAhE5Bzg48BnnG1xXy0xKhYLD/mPO0AH3q48+sOr+2g90BWyZUaYCPC1vzw1tME0NwJfBlap6kYRmQU84aslRsVi1UPBkmkB25bdnWzdd9i8hSpDgY07D/p+3oKEQFWfJNVKGhGJAftU9X/7bo1RsVh4qDxkW6fwh1f3saOjyxaHVQGv7u70/ZyFVg2tAK4D+oE1wBgRuVNV/9l3i4yKxMJD4ZBJGJY9+Rov7zxId3+SQ0f7OBriIja3o2lSGdSTKS6QiMeoiQm9SaXbFKxg1mxr971qqKCmcyLygqrOE5GPA/OBm4EWVT3DN0sKxJrORZfhNsJyuHDHo5v4+do3SKqSSKQKBXv7k5HpQhrj7fbEg7YLjEjE6FdMKNJYXML8Dz+aztU4Q2YuJTV3uFdEhksS3vAJCw9Fk5svnsvNF889ZntLazu3rlrPa3sPUZeIDbRYLrdIZLvEJxVryZEFt1OpXxQqBHcD24AXgadEZAbwVs5XGFWHhYcqiwUzGvn1jedl3Ocucvvjm28Rj8kgT6LzSF9Ftt4eTkz0ea1JyfMIRCShqllvG0TkBOAeYDKpZPdyVb0z7RgB7gQuBrqAT6nqulzva6GhaGPhoerAW720o+MIh3r6ONqTpLc/SVJTuYGYzTwIBAEeuP7cov+vhhwaEpGxwFcB9/bhSWApkKuOqQ/4O1VdJyINQIuIPK6qL3uOuQiY7XwtAr7nfDcqFAsPVQfpSepseNtyQ2royo6OI3T3p0TDKx5GfiaMqmH5Ne8KbR3Bj4ANwBXO808A/wFcnu0Fqvom8KbzuFNENgHHA14huAS4x5lR/KyIjBORqc5rjQrEDQ/19iUDWQFpVBaFCgakROPZrfsHPjPuyurOo6nAw8iaGL1J5XB3f2D2Rp32rmDauxUqBCep6kc8z/+hmBYTIjITOBNoTtt1PPCG53mbs22QEIjIEmAJQFNTU6Fva4TAghmN3PvZs1m5rm1Y9HUxyke6aLgrq9Pxlsge6umjt0+pjQ/OY/T2KTGg13E13DLVSg9dJRUeXNcWmkdwRETeo6q/BxCRd5MaNp8XERkNrARuVNWSEsyquhxYDqkcQSnnMMrLg+va6OlLsnJdm+UJDF/xtt8YKt5cR7nHjDbUxYnFhaM9SfqSSVRT4pSvUjYIGwsVguuAe5xcAUA78Ml8L3JKTlcC96rqgxkO2QGc4Hk+3dlmVDDPbt1PT1/S8gRG5Mk2f9pLpmHyfsyn7swQ4nJzJeNGJuhNKkd7+gd5LSJw2rSxx7xuqBTaYuJF4E9EZIzz/C0RuRF4KdtrnIqgHwKbVPXbWQ57GPiCiPyMVJL4oOUHKh8rIzUqkWLyGfB2iOr1vYcGRKKjq4eW7R30DzH73XEkc0GmKnztV/738hpK+eh2Vc0asBeR9wBPA+t5e83IV4AmAFVd5ojFd4ELSZWP/pWq5qwNtfLRysDKSI1qxU16dx7pZfXW/dQlYgOexBsHuti0a2i9ggT40p/P4Yb3nlzc63xYWZzNnqw4+YR8xyhwwxBsMCKKlZEa1Uo+z2JF83buX7N9kEBA4SIR5jyCTFjS1siKlZEaRmYWL2pi8aLMwZRsIuEyqaGOy+dP9/2mKmdoSEQ6yXzBF2Ckqg5FSErCQkOVw4rm7fx6w5tcdNrUrB98wzDKQ8mhIVVtCMYkY7jT0trO0kc20tOXZM22AzaoxjAiTKHD6w2jKDKVkBqGEU1MCIxAcHMEMbASUsOIOCYERiC4c4xjMRmYY9zS2h62WYZhZMCEwAiMTCWkhmFEDxMCIzDc8FBcgql9NgzDH8pe/mlUD9aJ1DAqA/MIjMB5cF0b9z23nY//4FnLExhGBDEhMALFykgNI/qYEBiBYmWkhhF9TAiMQLEyUsOIPiYERuBYGalhRBsTAiNwrIzUMKKNlY8agWNlpIYRbQLzCETkRyKyR0Q2ZNl/vogcFJEXnK/bgrLFiAZWRmoY0STI0NCPSY2gzMXTqjrP+VoaoC1GyFgZqWFEl8CEQFWfAg4EdX6jsrA8gWFEl7BzBOeIyIvATuBLqrox00EisgRYAtDUZJOuKhHLExhGdAmzamgdMENV/wT4d+ChbAeq6nJVXaiqCydNmlQu+4wAsDyBYUSP0IRAVd9S1UPO40eBGhGZGJY9RvBYnsAwokloQiAiU0REnMdnObbYlWEYY+0mDCOaBFk+eh+wGpgjIm0i8hkRuU5ErnMO+SiwwckR/BtwlapqUPYY4WPtJgwjmgSWLFbVq/Ps/y7w3aDe34gmmdpNLJjRGLZZhlHVWIsJo6xYGalhRI+wy0eNKsPKSA0jephHYISClZEaRnQwITDKzrNb99Pdmyoj7em1MlLDCBsTAqPsNNbX4paHJZ3nhmGEhwmBUXbau3qIOQmCmKSeG4YRHiYERtnxLiyL2cIywwgdEwKj7NjCMsOIFiYERih4F5Z19yZZua4tbJMMo2oxITBC4exZE0g4iQIFHmhpM6/AMELChMAIhQUzGvnYwhMGFpX191sZqWGEhQmBERqXz59OXY11IzWMsDEhMELDksaGEQ1MCIxQydSN1DCM8mJCYISKDasxjPAJcjDNj0Rkj4hsyLJfROTfRGSLiLwkIvODssWILhYeMozwCdIj+DFwYY79FwGzna8lwPcCtMWIMBYeMoxwCUwIVPUp4ECOQy4B7tEUzwLjRGRqUPYY0cXCQ4YRLmHmCI4H3vA8b3O2GVWGhYcMI1wqIlksIktEZK2IrN27d2/Y5hgBYC0nDCM8whSCHcAJnufTnW3HoKrLVXWhqi6cNGlSWYwzyou1nDCM8AhTCB4GrnGqh84GDqrqmyHaY4SI23LCpc+SxoZRNgIbXi8i9wHnAxNFpA34KlADoKrLgEeBi4EtQBfwV0HZYlQGp04bO/DYJpcZRvkITAhU9eo8+xW4Iaj3NyoPd3JZUkGADTsPhm2SYVQFFZEsNqqD9DzB/WveYEXz9nCNMowqwITAiAzpeYL+pHLbLzdY0tgwAsaEwIgUl8+fPuAVQEoMrJTUMILFhMCIFAtmNLL0ktOIO1pgISLDCB4TAiNyLF7UxFVnNQ08txCRYQSLCYERSSxEZBjlw4TAiCQWIjKM8mFCYESWTCGiWx9ab2JgGD5jQmBEmvQQUVLhllUmBobhJyYERqRxQ0Ti2abALeYZGIZvmBAYkWfxoiYuOGXyoG2qWJjIMHzChMCoCK79s5NIxGXQtqSJgWH4ggmBUREsmNHI/UvO4YJTJiMePTAxMIyhY0JgVAwLZjTy/WsWcvulpxNLE4NbVq1nyT1rbdGZYZSACYFRcSxe1MTX08RAgcde3s3Hlj1j3oFhFElg8wgMI0gWL0qtL7j1ofUk9e3tSYWvrFrPQ8+3MXtyA5fPn86CGY0hWWkYlUGgHoGIXCgim0Vki4jcnGH/p0Rkr4i84Hx9Nkh7jOGF6xlIhn3PbWvn3ubtXHH3avMQDCMPQY6qjAN3ARcAbcAaEXlYVV9OO/R+Vf1CUHYYw5tsnoFLf1K5ZdV6nnt9P7MnN3D2rAnmIRhGGkGGhs4CtqjqVgAR+RlwCZAuBIYxJBYvamLOlAaWPfkav9u0+xhBUOChF3YCkIgJSy85bUBADMMIVgiOB97wPG8DFmU47iMich7wCvC3qvpG+gEisgRYAtDUZP/AxrG4FUUtre2sXNfGlt2dPLft2AqiPsdDeOj5NsbV1zKpoc7yCEbVI6kZ8gGcWOSjwIWq+lnn+SeARd4wkIhMAA6pareIXAtcqarvy3XehQsX6tq1awOx2Rhe3PHoJpY9tTXvcTGBr196unkJxrBGRFpUdWGmfUF6BDuAEzzPpzvbBlDV/Z6nPwC+FaA9RpVx88VzAVj+9NaM+QMXqzQyqp0gPYIEqXDP+0kJwBpgsapu9BwzVVXfdB5fBtykqmfnOq95BEaxtLS28+zW/XQe6eX7T2+lP89HPiawcEajiYIxrMjlEQQmBM4bXwx8B4gDP1LV20VkKbBWVR8WkW8AHwb6gAPA9ar6x1znNCEwhoI3h7BmWzv5Pv0CXHDKZK79s5NMEIyKJjQhCAITAsMvWlrbs1YapWNeglHpmBAYRg68XsLa1va8ogAwd0oDJ4yvt6ojo2IwITCMAnG9hN++vDtv2MjFvAWjEjAhMIwiKcVLcDFvwYgiJgSGMQRcUXi+tZ1NuzqLeq3rLdjiNSNsTAgMwydWNG/n/jXbqUuk+jUW6y0I8K6ZJgxG+TEhMIyAGIq3AClhmNY4kuPHjmBcfS2ACYQRCCYEhlEGhuoteIkJzJncQG9/kvGjagc8iFOnjaW9q8e6qBpFY0JgGCFQ7OK1YogBUzN4EiYURjbC6jVkGFXNghmNAxdjVxT2dXYD0NHVMyRxSAI72o+wo/1Ixv0xYKGTi3Dp6OrhwOEexo+qZfbkBhMNYwDzCAwjJNw1C6/vPTQQ/uno6hlSSKkU4gLvSAtDefEKSPo+y2dUDhYaMowKwhtSci/AAC3bO+gvp0IUiLcSyiWXeGTbl237pIY6GuoSbHzzLS46bSpzpjTw7Nb9WT0Zt8mgeTqDMSEwjGGAe4FrrK/lic17QvckwiImoJr67l7o0wUzmVSEY/MoLn4KVSn7wsjnmBAYRhWQnodIx70g1cRjvLLnUCS9i2okXz4n3TsqNRRnQmAYxiC83sWGnQcHhaEKuaMdarLbKJ3aRIz7Pnd20WJgVUOGYQzCW9FUKrk8EL9CLx1dPZHNjYRFb1+SZ7fu9zWcZEJgGEZJ+CEmhZDuvQhw6rSxbNh5kH2d3ccIiBt/T8+jeAkzRzDUfE5NIsbZsyaU9uIsBCoEInIhcCepCWU/UNU70vbXAfcAC4D9pIbXbwvSJsMwKotSBWfxoqYArPGHQvM5fuYIchGYEIhIHLgLuABoA9aIyMOq+rLnsM8A7ap6sohcBXwTuDIomwzDMKJAubypQokFeO6zgC2qulVVe4CfAZekHXMJ8BPn8QPA+0VEArTJMAzDSCNIITgeeMPzvM3ZlvEYVe0DDgLHBL9EZImIrBWRtXv37g3IXMMwjOokSCHwDVVdrqoLVXXhpEmTwjbHMAxjWBGkEOwATvA8n+5sy3iMiCSAsaSSxoZhGEaZCFII1gCzReREEakFrgIeTjvmYeCTzuOPAv+tlbbCzTAMo8IJrGpIVftE5AvAb0iVj/5IVTeKyFJgrao+DPwQ+KmIbAEOkBILwzAMo4xUXIsJEdkLtJb48onAPh/NCZpKsreSbIXKsreSbIXKsreSbIWh2TtDVTMmWStOCIaCiKzN1msjilSSvZVkK1SWvZVkK1SWvZVkKwRnb0VUDRmGYRjBYUJgGIZR5VSbECwP24AiqSR7K8lWqCx7K8lWqCx7K8lWCMjeqsoRGIZhGMdSbR6BYRiGkYYJgWEYRpVTNUIgIheKyGYR2SIiN4dkw49EZI+IbPBsGy8ij4vIq873Rme7iMi/Ofa+JCLzPa/5pHP8qyLyyUzv5ZO9J4jIEyLysohsFJG/iarNIjJCRJ4TkRcdW//B2X6iiDQ7Nt3vrHJHROqc51uc/TM95/qys32ziPy537Z63icuIs+LyCMVYOs2EVkvIi+IyFpnW+Q+B573GSciD4jIH0Vkk4icE0V7RWSO8zt1v94SkRvLbquqDvsvUiubXwNmAbXAi8ApIdhxHjAf2ODZ9i3gZufxzcA3nccXA78GBDgbaHa2jwe2Ot8bnceNAdk7FZjvPG4AXgFOiaLNznuOdh7XAM2ODT8HrnK2LwOudx5/HljmPL4KuN95fIrz+agDTnQ+N/GAfr9fBFYAjzjPo2zrNmBi2rbIfQ48tv0E+KzzuBYYF2V7nfeLA7uAGeW2NZAfKGpfwDnAbzzPvwx8OSRbZjJYCDYDU53HU4HNzuO7gavTjwOuBu72bB90XMC2/5LUoKFI2wzUA+uARaRWYSbSPwekWp+c4zxOOMdJ+mfDe5zPNk4Hfge8D3jEee9I2uqcexvHCkEkPwekmle+jlMME3V7Pef/IPCHMGytltBQIbMRwmKyqr7pPN4FTHYeZ7M5lJ/FCUecSepOO5I2O6GWF4A9wOOk7pA7NDXrIv19s83CKNfv9zvA3wNJ5/mECNsKoMBjItIiIkucbZH8HJDyjvYC/+GE3n4gIqMibK/LVcB9zuOy2lotQlARaErKI1fPKyKjgZXAjar6lndflGxW1X5VnUfqbvss4J3hWpQZEfkQsEdVW8K2pQjeo6rzgYuAG0TkPO/OKH0OSHlN84HvqeqZwGFS4ZUBImYvTj7ow8Av0veVw9ZqEYJCZiOExW4RmQrgfN/jbM9mc1l/FhGpISUC96rqg5Vgs6p2AE+QCq+Mk9Ssi/T3zTYLoxy2vhv4sIhsIzXC9X3AnRG1FQBV3eF83wOsIiW0Uf0ctAFtqtrsPH+AlDBE1V5ICew6Vd3tPC+rrdUiBIXMRggL70yGT5KKw7vbr3GqBM4GDjqu4m+AD4pIo1NJ8EFnm++IiJBqFb5JVb8dZZtFZJKIjHMejySVy9hEShA+msXWTLMwHgaucip1TgRmA8/5aauqfllVp6vqTFKfxf9W1Y9H0VYAERklIg3uY1J/vw1E8HMAoKq7gDdEZI6z6f3Ay1G11+Fq3g4LuTaVz9agEh9R+yKVbX+FVNz4lpBsuA94E+glddfyGVKx3t8BrwK/BcY7xwpwl2PvemCh5zyfBrY4X38VoL3vIeWSvgS84HxdHEWbgTOA5x1bNwC3Odtnkbo4biHldtc520c4z7c4+2d5znWL8zNsBi4K+DNxPm9XDUXSVseuF52vje7/TxQ/B573mQesdT4PD5GqpImkvcAoUh7eWM+2stpqLSYMwzCqnGoJDRmGYRhZMCEwDMOockwIDMMwqhwTAsMwjCrHhMAwDKPKMSEwjCyIyC2S6mT6ktMZcpHTGbI+bNsMw0+sfNQwMiAi5wDfBs5X1W4RmUiqi+UzpGq394VqoGH4iHkEhpGZqcA+Ve0GcC78HwWmAU+IyBMAIvJBEVktIutE5BdOXya3f/+3JNXD/zkROdnZ/jER2SCpuQlPhfOjGcZgzCMwjAw4F/Tfk2pp/VtSMwCedPoDLVTVfY6X8CCpFb2HReQmUquBlzrHfV9VbxeRa4ArVPVDIrIeuFBVd4jIOE31RTKMUDGPwDAyoKqHgAXAElItje8XkU+lHXY2qeEwf3DaX3+S1FARl/s8389xHv8B+LGIfI7UIBLDCJ1E/kMMozpR1X7gf4D/ce7kP5l2iACPq+rV2U6R/lhVrxORRcBfAC0iskBV9/truWEUh3kEhpEBSc2Sne3ZNA9oBTpJje0EeBZ4tyf+P0pE3uF5zZWe76udY05S1WZVvY2Up+FtHWwYoWAegWFkZjTw705r6z5SHR2XkGoX/F8islNV3+uEi+4TkTrndbeS6nIL0CgiLwHdzusA/tkRGCHVXfLFcvwwhpELSxYbRgB4k8ph22IY+bDQkGEYRpVjHoFhGEaVYx6BYRhGlWNCYBiGUeWYEBiGYVQ5JgSGYRhVjgmBYRhGlfP/AV/2kGjJBFS8AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.plot(loss_monitor.loss, '.')\n",
    "plt.xlabel('Steps')\n",
    "plt.ylabel('Loss')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过如下方法打印量子嵌入层的量子线路中的参数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.06950325e-03, -1.62345007e-01,  6.51378045e-03,  3.30513604e-02,\n",
       "        1.43976521e-03, -8.73360550e-05,  1.58920437e-02,  4.88108210e-02,\n",
       "       -1.38961999e-02, -8.95568263e-03, -9.16828722e-05,  6.78092847e-03,\n",
       "        9.64443013e-03,  6.65064156e-02, -2.27977871e-03, -2.90895114e-04,\n",
       "        6.87254360e-03, -3.33692250e-03, -5.43189228e-01, -1.90237209e-01,\n",
       "       -3.96547168e-02, -1.54710874e-01,  3.94615083e-04, -3.17311606e-05,\n",
       "       -5.17031252e-01,  9.45210159e-01,  6.53367564e-02, -4.39741276e-02,\n",
       "       -6.84748637e-03, -9.54589061e-03, -5.17159104e-01,  7.45301664e-01,\n",
       "       -3.10309901e-04, -3.35418060e-02,  2.80578714e-03, -1.21473498e-03,\n",
       "        2.32869145e-02, -2.02556834e-01, -9.99295652e-01, -2.33947067e-05,\n",
       "        6.91292621e-03, -1.37111245e-04,  1.10169267e-02, -2.61709969e-02,\n",
       "       -5.76490164e-01,  6.42279327e-01, -1.17960293e-02, -3.99340130e-03,\n",
       "        9.62817296e-03, -2.04294510e-02,  9.17679537e-03,  6.43585920e-01,\n",
       "        7.80070573e-03,  1.40992356e-02, -1.67036298e-04, -7.76478276e-03,\n",
       "       -3.02837696e-02, -2.40557283e-01,  2.06130613e-02,  7.22330203e-03,\n",
       "        4.16821009e-03,  2.04327740e-02,  1.80713329e-02, -1.01204574e-01,\n",
       "        1.14764208e-02,  2.05871137e-03, -5.73002594e-03,  2.16162428e-01,\n",
       "       -1.32567063e-02, -1.02419645e-01,  4.16066934e-04, -9.28813033e-03],\n",
       "      dtype=float32)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "net.embedding.weight.asnumpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 经典版词向量嵌入层\n",
    "\n",
    "这里我们利用经典的词向量嵌入层来搭建一个经典的CBOW神经网络，并与量子版本进行对比。\n",
    "\n",
    "首先，搭建经典的CBOW神经网络，其中的参数跟量子版本的类似。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CBOWClassical(nn.Cell):\n",
    "    def __init__(self, num_embedding, embedding_dim, window, hidden_dim):\n",
    "        super(CBOWClassical, self).__init__()\n",
    "        self.dim = 2 * window * embedding_dim\n",
    "        self.embedding = nn.Embedding(num_embedding, embedding_dim, True)\n",
    "        self.dense1 = nn.Dense(self.dim, hidden_dim)\n",
    "        self.dense2 = nn.Dense(hidden_dim, num_embedding)\n",
    "        self.relu = ops.ReLU()\n",
    "        self.reshape = ops.Reshape()\n",
    "\n",
    "    def construct(self, x):\n",
    "        embed = self.embedding(x)\n",
    "        embed = self.reshape(embed, (-1, self.dim))\n",
    "        out = self.dense1(embed)\n",
    "        out = self.relu(out)\n",
    "        out = self.dense2(out)\n",
    "        return out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "生成适用于经典CBOW神经网络的数据集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train_x shape:  (58, 4)\n",
      "train_y shape:  (58,)\n"
     ]
    }
   ],
   "source": [
    "train_x = []\n",
    "train_y = []\n",
    "for i in sample:\n",
    "    around, center = i\n",
    "    train_y.append(word_dict[center])\n",
    "    train_x.append([])\n",
    "    for j in around:\n",
    "        train_x[-1].append(word_dict[j])\n",
    "train_x = np.array(train_x).astype(np.int32)\n",
    "train_y = np.array(train_y).astype(np.int32)\n",
    "print(\"train_x shape: \", train_x.shape)\n",
    "print(\"train_y shape: \", train_y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们对经典CBOW网络进行训练。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch:  25 step:  20 time: 0.023, loss is 3.155\n",
      "epoch:  50 step:  20 time: 0.014, loss is 3.027\n",
      "epoch:  75 step:  20 time: 0.022, loss is 3.010\n",
      "epoch: 100 step:  20 time: 0.021, loss is 2.955\n",
      "epoch: 125 step:  20 time: 0.021, loss is 0.630\n",
      "epoch: 150 step:  20 time: 0.022, loss is 0.059\n",
      "epoch: 175 step:  20 time: 0.023, loss is 0.008\n",
      "epoch: 200 step:  20 time: 0.022, loss is 0.003\n",
      "epoch: 225 step:  20 time: 0.023, loss is 0.001\n",
      "epoch: 250 step:  20 time: 0.021, loss is 0.001\n",
      "epoch: 275 step:  20 time: 0.021, loss is 0.000\n",
      "epoch: 300 step:  20 time: 0.018, loss is 0.000\n",
      "epoch: 325 step:  20 time: 0.022, loss is 0.000\n",
      "epoch: 350 step:  20 time: 0.019, loss is 0.000\n",
      "Total time used: 8.10720443725586\n"
     ]
    }
   ],
   "source": [
    "ms.set_context(mode=ms.GRAPH_MODE, device_target=\"CPU\")\n",
    "\n",
    "train_loader = ds.NumpySlicesDataset({\n",
    "    \"around\": train_x,\n",
    "    \"center\": train_y\n",
    "}, shuffle=False).batch(3)\n",
    "net = CBOWClassical(len(word_dict), embedding_dim, window_size, hidden_dim)\n",
    "net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')\n",
    "net_opt = nn.Momentum(net.trainable_params(), 0.01, 0.9)\n",
    "loss_monitor = LossMonitorWithCollection(500)\n",
    "model = ms.Model(net, net_loss, net_opt)\n",
    "model.train(350, train_loader, callbacks=[loss_monitor], dataset_sink_mode=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "打印收敛过程中的损失函数值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEGCAYAAABvtY4XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA6k0lEQVR4nO29e3iU9bnv/fnNTBJOAcZwCBiSGLUpELo0QRLUonYVV3W5q0JtNX1X27dVpKvv9S6vrvVurbbubmy7XWvv9upa++0WD3V3dW2gqAi2bl31hOKJAEEwQIxiICEcEsAJBAI5zPPbfzzPM3lm5pnJJGSSOdyf68rlzHO8E8bv85v7d/++t9JaIwiCIGQenrEOQBAEQUgOIvCCIAgZigi8IAhChiICLwiCkKGIwAuCIGQovrEOwMm0adN0aWnpWIchCIKQNtTX15/QWk9325dSAl9aWsqOHTvGOgxBEIS0QSnVEmufpGgEQRAyFBF4QRCEDEUEXhAEIUMRgRcEQchQROAFQRAyFBF4QRCEDCWlyiQzhbV1rTz9TjOd5/sAONcb5HxvEKVAA1qDx3ptGOY5Xs/I7DO0+eNVgBr56wFMyPGSl+sFIM/n5eIp4wD47GwvF03MZeqE3LC/x/T8PJZVFlFV4h+xv7EgCIMjAj8EHn2pkd+/f5Dz/UZIGCNFMq75smOnEXFgvzGy+4J64H4jfb2uniBdPcHQvsOBcwMHHj+LG2vrWrl3SRkP3DzXdb8gCCOPCLwL9gj8+NkeunuCBA2N1uHibcQRayEaDaze0syx0+f59Z1XjnU4gpAViMBbrK1r5TebP+HY6fMEjcGPF4bHpl1H2HWok2sumyZpG0FIMlkt8I++1MiauhbO9gZHbRTutdI56ZiD14zMt5WDJ7s5eLKVddta+fLcmdx73aUi9IKQBJIu8EopL7ADOKy1viXZ90uEtXWtrPrTXs73D3+oHk9A83weZk4eR45XhSYdM2Wisb4lwIadbexv7+Jw5zl6XL7unOsNctaRo4+FoeGVfe281tjOz25bQG11cTJCFoSsZTRG8H8HNAKTR+FeMbmQFIzCHNnm+rxUzJ7M/TfNTXuhHi5VJf6Efvf6lgCr3/qU1xvbBx31Gxoe3NjApg/auHxmfkY8CAUhFVDJbLqtlCoC/hX4OfDDwUbwCxcu1CPtJrm2rpVfvdrEiTO9CZ/jVZCXI2I+EthC/1pjO4l+1LwexSO3VsiIXhASQClVr7Ve6LovyQL/HPBfgHzgH9wEXim1AlgBUFxcXNXSEtP5ckjUtwT4wf+q51hXT0LH53oVM/Lz+NsbLhdhSQJr61p5cGNDwscrYOk8yc8LwmDEE/ikpWiUUrcAHVrreqXU9bGO01o/ATwB5gj+Qu871BF7acEEfvn1K0REkkxtdTFvNnXwyr72hI7XmPn5N5o6WL9isfz7CMIwSGYO/hrgq0qpm4FxwGSl1P/SWv9fI32j+pYAj77cyIeHOukJJvaMmDcrn0duW+AqHPZE4glr9J8pE6Rjzb3XXcrrH3UQdEnKK+Dzhfl81N4VlsrpD2o27GyTv70gDIOkpmhCNzFH8K4pGifDycHXtwRY/th7ww8ujfF5wD8hlyuL/WOeynA+FDu7e0O2BZGTpmvrWvnxpgbXidfa6mIqZk/hoY0NYYvKbpw3kye+5foNVBCynjHLwTsCuJ4kCfwlD/zv+PYAQkqQn+flRzfPo7a6OKbI25OrQNh+mXQVhNjEE/hRcZPUWr+ZrBp4Eff0oKsnyIMbG/jWb+uorS7mZ7ctMNcSOAgamodf2EN5YT53LSqO2l7fEhjlqAUhvUl7u+C0/wWyjC2fnAiJ/J2LokfkQUOztfkkyyqL8HlU2PYNO9tGM1RBSHvSXh+bH/3rsQ5BGCJbPjnBF376Z+bPnhIm4mBOttaUFVBV4mfVrRWhUb4Gnqtvk1G8IAyBjPCiuXHezITL72zmFuaTP87HZ2d7KZs+acwnKWOxtq6V9dtbyfN56DrfT1ugG62hJ2jQl2DFUCpy+nw/D25s4KIJOQS6+0KpNgNoOtZFVYmf2upi9h45xdq6VjTQ129IRY0gDIFRmWRNlOGuZB2JShoFXFXqDzWrcFaCOBtY2NtT+aGQKjz6UiOrtzQP+bzLpk/ktb+/HjD/be964n16rYdZrs/Duntq5O8uCBZjstBpNKkq8VOYn5fwqlU3NLDtoMvX/xgNLPYfP8ur+9oTeijE2zecc0brXhda///AzXNZOr+Q/3fdTg53nk/4vP3Hz7K2rpXa6mKqSvzcsXCOjOIFYRhkxAge4KGNDaypa417jM8zYH8rJIZHQfnMfPqCRtSDIdEHQH1LgG88/j79Q/jDf/Hyafzb96pD58soXhDcGfMyydFgWWXRoMf0G5a3uce0KFhU6ufiqeNQg56ZvRgaGo91sf/4WbYdDPDKvvbQz5q6VpY/9h5fX/1e3MlPe8LUM4Q/9O5DnaFr2qN4+/Rg0GBr88kL+K0EITvIiBQNmCKwqNTvnmaJIGiYTSdaTnZz75Iyls4vDLMmgPhpjsOd5zjSeV5q8C22HQzwtcfei2sOZi9S+smmBhKZGz59vp87Vr/HsyuvpqrEz7LKIjbsbKO3z0AphT8ixSQIQjQZk6KBkbMt8CooL4ztVWPfK9GHQrx9qZqD7+zuZUdLYMjpLI8ibpemRFJpTpw2BWvrWnn4hT0YWpPr87DmbknTCELGT7LaDGUUH4+ghn1Hu4b/sIgxMRt333DOGe17JYDdpen1jzpc7QWWVRaxfvuhhPPxrzW2U98SoKrET6C7F0NrDG1Otm5tPikCLwhxyJgcvM39N80d6xAEzJWnP97UwNqI0XpViZ+7r70k4esYmtAK1pqyAnJ9HjwgaRpBSICME/iqEj8rl5SNdRgCpjj/ZFND1ARs/vic0ISrvf4gHvvbuwDz3/bhW+bj8SgMrVn14l5Z2SoIccg4gQez/rpo6riY+xXmStaJud7RCypLCWr44fpdYdtqygpCFgUa2N12iiWXT4t5jR0tgZCQu6VpBEFwJyMFHuBvb7g85j6NWfp3vt/gF7cv4OCjf82G71/N0nkzmTYpl5xIm0Phgmj5rJulv3wz9D6y7LGv36DoogmUXDTB9XxDw+NvfQoMpGm8CnJ8HmrKCpIcvSCkLxlVRRPJt35bx5ZPTsQ9RgE/v32BeI3HwW6cve/IKcsDx+DM+X6CxtCus+Tyafw+zuKln/6H+TH7tnoUoZLJtXWtvLznKDdVzJJ/NyHrGfOGH4ky0gL/m837+a9/bkro2EWl/qjuQ0J87FaJ24dQtbRySRkP3GxOhD+0sSFkQeBV8MMby9l9qDOmcdyNVp39N5/aSm+/IaWSgkCWrGR1o6aswPUXzHVJwWw7GGBNXStfe+w9vvyrt6KqP4Roqkr8PLvyajZ8/2rmFuYndM7qLc2hfPqyyiLycsKrYu697tKY577+UQfP72yjt9+QHLwgJEBGC3xViZ8VLhU1vUEd055AA/s7zvDgxgZu+vUWVvx+Bw9tjK4EEQaoKvHz8n1LmDcrMZF/9OXG0HmRVTEAhZPzXM8zDI0GKZUUhATJqIVObuSPz3HdrjH7hJ7pCca0HGg81kXjMbNEb9221ijTrQt1W8w0HrltAXesfm/Q1a/bDwbiLl667YqLXW2GNTA5z8fDt8wPrWhd9eJeygvz5d9AEFzIeIGvKSsg16tCk3lOunqCgJl/B1N4YmmTbboFhK32XLetlYWWuAxmD5DpOf6qEj93LSpOyIrAtvy1q2KcHjM/uOEyfvfeQc73R8/iPvnOAe68ao6saBWEBMh4gbdL8uKJzraDAX5x+wLuv2nukCcNDR3hIx/HHmDbwUDogWD7vYyl3/twrjfYtxbTiqAVF20Ow/bwsdM0kSPy/HE+zp/pjTov6EjT9PUbUiopCHHIeIGHxPxPHtzYwC9uX8CzK68OMxIbrulWLKIeCJA6XjMJXs8tXWUzPT+PL31+8BaKnd0D4u2Wprmy2B/zGpPzfKy5u4YNO9vE6lkQ4pAVAm/7kQ9mVfvgxgY2fdDG/TfN5Re3LwhttwV/f3tXaEQLjKjwpxOx0lU2yvqJ96dx5uHtNI1zRO6fkBtT4J965wDFBRNDFTUbdrZJuaQguJAVAg+EGjgPlh/edtC0HJ4+KZcri/0h21s38Ygc6cdLc+R4PTS1d2XFAyGRX1EDq9/6lCe/tZCqEn/UiDzQHZ2esQkampf3HI0qlxSBF4RwskbgYWhWtcfP9IY6F5UWTOCay6ZF5Z5jCX8sIj3k0y0HP9LpqgPHz4S9d47IH75lPh7l3l5RAwUTcyUPLwiDkFUCn2iqJpKDJ7s5eLKVNXWtTB3vQyswDNOw7P6b5iYs8kN9IKQibukqZ9PxeJVIkTSfOBtK02xtPhk2Ig909/Kz2xbEtC548cOjrLq1gj1HTkkeXhBikFUCD+6pGgUUXzSBls+6Bz2/81x/6LWdzpk63ofP56EvaHC+1ywfGZ/jwecbWEfWFzQwDCiaOp45EaZakSPlVK6vH+whtbaulR9vakholG/ogTSNWx4eQClwc9MwtGbPkVOShxeEOGSdwEN0qkYDbYFufnH7AjY3dbDtwElOOYR8MDpdju2JUSfoXDwVhWPCck1dK5NyvfRZMbo9MOI9TIa6ry9o0NevmZjr5ZJpE4dds19bXUx5YT6r3/qU1xvbBxV6O03jloff2nzSVdwBfF4PCiQPLwhxyEqBt1M1D21sCKUTgho2N3XwpKP/569ebeKESy32aHGmNxh6HeuBMdL7unuDHD/TG/LmsVNStvg7J55jUVXi58lvLaS+JcD9z+1mf5zSzJbPukNpGojOw3sVrum0/qBBfp5P8vCCEIeMdpMcjBW/3xFWiqcU/Py2cOtgZ875cOc5eiyr3PN9Q/TKzTDmFuZTWeIfdJRf3xLga4+9FzMvr4B/+KtyfnDDZfxm835++UoThh5wl/RPyI2Zh/d5FKturSDQ3UtNWYGM3oWsJGuabg+Ve6+7lNc/6iBop2o0/HiTKSa2yMfKOa+ta+Xpd5rpPN8X2hYvNZJpDwU71bS2rpWllo2v29+pqsTP0nmxFz5poOuc+TeMlYf3enD1nje0Dom77SopIi8IA2T1CB7cJwUVxBWtC7nX+u2t5Pk8cUsVAQ53nuNMb/+I5tnj7TvXG+Rsz0BKaKh4PfDMvVfHXC8QbxT/xcun8W+ORiBbm0+GRuTxPP3tJiGrXtwr/vBC1iIj+DjYI3WnyGvglX3tvP5RB4/cWjFiXYNqq4tTugNR5MItOyWViPgHDfjxxgZevm9J1L6qEj9fKJrC7rZTrucWTMwNO9Yp0PHsgP+iaAqB7l6ZaBWEGGS9wIO7yIO5YvKhjeEpm0wmXgmkLf4ftARiVgE1Huvi0ZcaQx2bnCwuK4gp8C9+eJS/WVxKVYk/agQf6O6NueBpZ2snt19ZJBOtghADEXgLW8AjF0FpTI+aN5s6Rjxlk044xd/u0fqqS1798bebWTq/MOrvFMuXH8wHqZ1Dj2zHZ+fl3eYvDMPMwa+5uybsoSAIgklGd3QaKrXVxTyz8moumzEpat8r+9q5Y/V7fH31e1nf4ckug9zw/auZPik8haK16fUeSU1ZAb4YnzaNmYqJXM1qp1u+s7jU/URFSNTtidZs/ncRhEhE4COoKvHzj8u/gM+lb6tt9bumrpXlj5lin82CUlXiZ/XfLMQT8adav7016u9SVeLnG1fFTnPtOXIqNFr3KsLSLXuPnnY9R2t4de8x6lsCfPOprfzylSa++dTWrP43EQQnIvAuVJX4Wb9iMUvnzYwSLyfbDprVIV9f/V7W9m61uzg5CRqmBUEkyyqLYn7gnt1xCIA1d9fwwxvLw6ph5s+aHPP+T7zdLI24BSEGkoOPgXM1Zrxl95rwBh5r61qZ7R/PxVPGRZVCprLHzIWwrLKIP2xrDZu7eL2xPWyFKph/07IZk9jfcSbqGv1BMw//gxsuc83fx/KX1xrp8CQIMUiawCulxgFbgDzrPs9prf9Tsu6XLJxCP1gVCZhiczhwjsOBc67719a1clWpPyk2v2P1AKkq8fNIhPOjoQf6rjr57jWXuK5M9XlVSJgjK2lqygrIy3GfaPV5Fcsri1heWSQTrYIQQTJH8D3Al7TWZ5RSOcA7SqmXtdZbk3jPpOGsIrFXsX56/GzC1rg2kSP+MEag1V68dnoQ+4FxoQ+H2upi3mzqCFuxavveRx739LsHokbxQUPTZD04Iytp7InW1Vuao67XHzTPs6ugZEWrIAyQNIHX5hJZ+//iHOsndZbNXgD2gqXIhUFD8UJPFoO10wvhss+ZXhqOm+S9113KG00d9Fu5mjeaOqLSNOA+ijc0PPzCHr5x1RzXSpqYE63AT17YAyArWgUhgqTm4JVSXqAeuAz4jda6zuWYFcAKgOLi9FpMFLkwyM7XHzh+5oIaYYwVzvTStoMB1m1rZWGJP+H0T1WJny+VzwiN4vuD2jVNU1tdzL/vOcqWT06EbQ8aOmY+ff6sybwdcbyNIS38BMGVpAq81joIXKGUmgpsVEpVaK33RBzzBPAEmF40yYwn2dj5ejci2/U5GYkcPIx8E3C7LNRm3bZWfhbhthnJ9Py8sPduv299S4B390eLtVLEzKfHm2jN8Xm4qWIW2w9+JhOtguBgVKpotNadSqnNwFeAPYMdn4mMRru+eO30bNweGIl+wzA0PLQpvnXDssoi1u84FDdNs7X5pKvHu+175/a3ijXRetHEXP7hxvJQoxGZaBWEAZJZRTMd6LPEfTywFPjHZN1PuLCHiDO9lOP10NTe5V4W6mKpHBnDYGmamrICcr2K3giV15gNP2LZDj98y/ywJi0An53t5SdWPOWF+UP8rQUhs0nmCH4W8K9WHt4DPKO1fjGJ9xMugMj0kvPbQOTo3hhE5CPTNJFrxapK/KxbsZhHX25ke0RF0cftXaH7R47G32zqcP2WEbTi8Xk99AdlklUQbJJZRfMhcGWyri8kFzdzsdca20NplHgi70zT+LyKZZVFrte/vnxGlMDXtwRYW9fqWhHTfvp8zHjtyVWNTLIKgo1YFQiDYo/uf37bgjDrBlvk19a1Rp3jUeaBWhOqb4+kpqwganRvaNPLxs16IJ6Xjc+ryHHxsRGEbEasCoSEcfPNt+vXywvzQyPmrc0n6bMaegcNHbXfycJSf9Qofu/R0/i8HoLB8IqY2upiWk+ejVrwlD/Ox+KyAq4vnyH9WQXBgQi8MCTcRD5ohE+k1pQV4PUo+q0DDK2jUia2A6Sb/YA2NF9bNIeLp46PEmu3csmu8/28sq+dNz8+zrp7JPcuCDaSohGGTG11MT+7bQG2o7IGnqtvCzlpVpX4WXVrBV5lTrD6PCoqZWJ7v7uhgYrZU1yNx/wTcmOWc/b1G2zY2cZvNu/POldPQXBDBF4YFrXVxdzpsAnuj7DpLS/MR3kUGnCTcdv73Q1Dm7YDbiId6O6NytvbKGU+aMQXXhBMROCFYTN/9pTQa4PwBtkbdraFFjvZtfBOqkr8rLm7htrqYlfB7ukzXDtDxRrBK+DLc2fSHxRfeEGwkRy8MGycDbE9ynxvEynabiJul2JWzJ4SZT6mgWd2HEJBmAeO8x5OPl+Yz7T8PNfJWUHIVmQELwwbO83iwSyLdI7gl1UWkevzoDDNw9xq4W02N3W4bu8PatbUtYalW2rKCnDppkjjsS7W1bWC1ty5qFgWOgkCIvDCBWDbB3g8CkPrsLx5VYmfdffU8OV5M5lbmB+zFh7gwIk4tsZAryPdUlXi5y/nznQ9TgP9hmb21PEi7oKACLxwgQS6ezG0ds17Nx3r4tV97exuO8WDG90XRNW3BAZcxmLgUeFVOPdedym5bsN4l2MFIZsRgRcuCGeaRkWkaV7eczTs2Mj3di38/ojmI0sun8a4HDO941Fw97WXRPV2XbdiMZdNnxh2nlKw6tYKACmVFARE4IULJF6a5qaKWWHHRr6PVQv/9v4TfGV+IV7LF+F37x+MEuumY118GvFguKrEz54jp7jrya1SKikIiMALI0CsNE1tdTG/uH0Bl02fyGUzJkWd5xz9O9EaXth9JGbqp74lwE82NUSVS+5oCbCuzt3HRhCyESmTFC4YW6hjdVOyUzB2KaRtd2DXwm9tPsmLu48M9JLFFHmvR4HWUamfWA1DnP71CjEdEwQZwQsXjC3UP7yxPKo8cbA8fFWJnx/ccBlFF00I227n3pXlafPTP+0NK5WMNckK4FXmQ0RKJYVsRwReGBGqSvzUlBWwtflkWN57sDy8zYyIJiFfnjuTrp5+gtawvLff4HlrZas9ybp03syoBVQeBY/ctoCf375AxF3IeiRFI4wIdkVMZJMOOx3z9DvNZplLDJZVFvFsfVto0rXTZcWqMytTVeLnijlTedVqDejcHujuZW1dq1gHC1mPCLwwItgVMc7JTaewxsrDw0B7vpsrCtm06wgA2w4G8HrA64GgAT4PLI9YDeuPaChuX2uH1WLQo5D2fUJWIwIvjAjxJlrd8vC2wDtH/pHrnYKGOdGq0Hg80dlEN18a50RrrIeNIGQLIvDCiOCsiIlMi9xUMYu3PzkR9t7GOfKPRAFa61CfVWdTkfqWALsOdcaNySPt+4QsRwReGDGcjbqdxMvDO0f+QFj548JSP7sPddIb1KGmInaa5ptPbaXHpRuUzdJ5M7lizlTJwQtZjQi8MGq45eGdI3//hFx++sc99Foqf/HU8XxuZj5rLA8bZ1OR3n4jZmcnn0ex8rpLRdiFrEfKJIURpb4l4OoDE68e3q6Fr60u5uYFA+mbTbuOcOiz7tB7u6lIrBWwYH6gv/T5GSPxqwhC2iMCL4wY9oSpmw9MovXwkXn1Dw+fwrKkCTUVsUf911w+LbTPxgBe3dcuPjSCgAi8MIK4lUra2L40X7x8Gr+4fUFYmaST4ogVraUXTSDX58FrlTzaE6ZVJX7u+/Ln8EUqPGa9fK804BYEycELI8dgnjS2qNvpGbda+FPn+sLOyR+fw5q7a1j91qd0nD5P07GuUG69qsTPHQvnhHL0kTxX30Z/0JBaeCFrEYEXRox4pZIAa+taQxOsdtlkbXVxWC185IB8/qzJocYhALvbwhdKRa6AtZk9ZRxHTp2XWnghq5EUjTCi2BOmbmIaa6LVmdrRGq4qHTj3qXcPsH57q+t59S0BNuxsw3DpCDUux4vPa6Z2pBZeyFZkBC+MGrEWPEWmdpyD+P6gjhqd31QxKzTq7+lzL5dsPnEWn0dx56JillUWyehdyEpE4IVRw5mDv6lilqsvfE1ZAav+tDfsvFyfh1/cviBsoZQ96o9VC29oCEoDbiHLEYEXRhx7wtQtDx9rotW5CnZxWQG7206FzllspVecC6VWLikLjfqVMn1rIsXe65XUjJDdJCTwSqmJwDmttaGU+hzweeBlrXXfIKcKWUYs22CbWBOtTk739Ie9//TEWfYePR22be/R02ErYH+yqSGqy9N1n5sOmA24xbJAyEYSnWTdAoxTSl0MvAL8DfC7ZAUlpC/xauEh/opWexXsia6esGPe+KiD+bMmh227qWJWaEI30N3r2sLvwPEz0oBbyGoSTdEorXW3Uup7wP/QWv+TUmpXEuMS0pTBauFjTbQ6R/4+j8KjBqx/g4bmdE8/OV5FX1CT41WUF+aH3dN5vM2nx8+G0jZSKilkIwkLvFJqMfBN4HvWNm9yQhLSmcFq4WNNtDpH/kFDUz4zP6wJ9/72rlD7vv6gDrMOrirxs+KLZaze0hw6XjGQk5cG3EK2kqjA3wf8CNiotd6rlCoDNictKiGtiWUbbFNemM+eI6fYe+QU9S2BUD9X58i/L2I4/tnZXnweFbIOfmbHoZB18NbmkyydXwjApl2HQUP7mR6Uhhyv4o6Fc6RUUshKlHZZJBL3BKU8wCSt9elBDx4iCxcu1Dt27BjpywopRH1LgLueeD9kCZzr87DuHnMi1ll9s/qtT8P6rdoNtl+J2Pb2J8dDaR0Dc3TvZOWSMh64ee5o/GqCMCYopeq11gvd9iU0yaqUWquUmmxV0+wB9iml/r+RDFLIDrY2n6TPIcLOiVjnKtiV112Kz/p0ehTcUD6D6fl5YdfqOH1+YEI3qKPEHYiqvhGEbCLRKpp51oj9NuBl4BLMShpBGBI1ZQXkeAfWqsbKjVeV+Ln72jI8yrQvWPXiXubPnkKupfpKwSXTJoacJnO8KsrHBmB8jleqZ4SsJdEcfI5SKgdT4P9/rXWfUipubkcpNQf4PTATc77rCa31P19IsEL6U1XiZ92KxWzY2YaCmLnx+pYAT71zIFQZ09NnEOju5btXl7J6SzNamw1BVi4pI398DjVlBby691jYRKtHwWuN7bzZ1CF5eCErSVTgHwcOAruBLUqpEmCw7779wN9rrXcqpfKBeqXUq1rrfcOOVsgIYk3COnPwW5tP0u+YaNWY3Zwi6+r3Hj3Nv32vGiBqn316b1Cztq6VDTvbxDZYyCoSEnit9b8A/+LY1KKUumGQc44CR63XXUqpRuBiQAQ+C4hnVwDmilZnqWTkCtjvLC6NOmfPkVNRdfTOBVD+Cbkx49FILbyQfSRqVTAF+E/AEmvTW8Aq4FTMk8LPLwWuBOpc9q0AVgAUF7t3+RHSi+HYFQS6e8NWwLpNjp7o6qG2upjWk2d5/G0zTfP0ewdZOr+QqhI/ge5e13i8ViJfauGFbCPRSdangS7g69bPaeB/JnKiUmoSsAG4z620Umv9hNZ6odZ64fTp0xMMR0hlhmNXYNfB2/7tN1XMItcbPmv65sfHqW8J0NXTj13d29tv8Phbn/KbzfvxT8hlXE70R7pi9mR+eGO5pGeErCPRHPylWuvljvf/ORGrAmtidgOwRmv9/DDiE9KQ4dgVuK2ALS/MZ9Wf9oacJYNB82ERObv/WmM7rzW2h1I7z+1s48SZgdF8nhWDiLuQbSQq8OeUUtdqrd8BUEpdA5yLd4JSSgG/BRq11r+6sDCFdGK4dgWRk69VJX4WlxXQcPgUWoeXVD6zvRW7D0hoMrXP4Mm3m6OMx7YfNFNGMoIXso1EBX4l8HsrFw8QAL49yDnXYNbKNzhG+w9qrV8acpRC2jGYXUFtdXGUTTCET842HesKK3v8zuLS0DW/cVVxWLNtBXg8KqzyxkYmWIVsJdEqmt3AXyilJlvvTyul7gM+jHPOO4DL0hNBMImstImcnC3yTwg7/n1HLn/+7Clh+64q9XPblUWuvvAKc6L1SOe5kPeNIGQDQ2q6rbU+7Zgo/WES4hGyBNuT5r/9uYm7nng/JPbOyVkifJKcvVkjK2Z2tnZSXpjPI7ctIGJuFg0ENazb1iq+8EJWMSSBj0BG58Kw2bCzLeQM2WvZ/0ZW0nz32rKwD2jjsS7WWmmZmrICfA5vAkNrtjafpLa6mGdWXs1FE3PC7hc0dMyqHkHIVC5E4IdmQykIDiJHB4qByVm7pLG2upgFReGpGLvEsqrEz6pbK/B5lJl/VypsoVPgbHg3SfMY8YUXsou4OXilVBfuQq6A8UmJSMgKllUW8Wx9W6iUcpnl7R45ORvZgNu5crW8MJ8bPj+DNz7qIGhoHn5hD2CueHX70HqU4uFb5ksOXsga4gq81jo/3n5BGC5VJX7W3eNeSumcfI1swG2/tydke/qMkJj3WyJfWTw16n4a0FrHXO0qCJlIomWSgjDi2KLu9IOPrKRZcnn46ma7Ibc9IRs5Uu83tOskqrTtE7IREXghaQxmOGZX0tiNtNetWBxVSTMtPw+fV4Waedh2Bc7VskBYaaRLKTwoJD0jZB0XMskqCDGxR+K/fKUpZmliIpU0FbOnhOXd+4MDC5bsCdk7FxWHJm3tydRItDYnaKVEUsgmZAQvJAU3w7HI0XO8SpqtzSfxT8hl1Yt7Od83UP9u6AFbYPt6G3a24fUqgtY3gevLZ4T1brV5+5MT1B34LNQDVhAyHRnBC0khciTulvteVllErs+Dwmy+7ayk+cENl4UshCPZ3NQBDHxLWFfXSr/1TQBlCnykE6VNb7/B8zvbRurXFISURkbwQlIYzHDMPmbdPTWh9n1O6lsCHO48h8/riRL5A8fPAO4TrX39BnuOnOKOhXPCvGqcyAIOIVsQgReSxmCGYzbP72yjt98ItdQDQpU0Po9iUamfbQcHcuctn3WHTbT29hnYjwANPLPjEKu+WoHPA5FfAHK9iuXWNwVByHQkRSOMKW65eue2oKG5rnwGN86bGTrHMHTYROvf/1U5i0oHHiT9Qc3GD9qiqmm8HsVPv1oh+Xcha5ARvDCm2J4yfUGN16NCufrIhiE1ZQW8+fFx+voNvN6BnL79LeFI57mwUf6OlkCkVxlBQxY6CdmFCLww9igFaOu/7vn7+pYAQcPMtweN8LxLfUsADeR4zQeFUlFGlCG6zvW57xCEDERSNMKYsrX5JP1BI9SUY4NV4WJX0tjifv+GDwlauh404PG3PgUGKmn+sK0VrbXVYNs8ThFdivnE280hR0pByHRE4IUxxWn7q4Hn6ttCi5HqWwI8uLGBu57cyv6OM2Hn7T1iGpCF5+vN/LzG/GB/oWgKnohPuKHh4Rf2yIInISsQgRfGlKoSP3csnBMaaduNtZ017m618Ic7z7O2rjW83t6ryPF58GC278vzeUKjfie2d7wgZDoi8MKYs6yyiLyc8EVRkTXubumWl/ccDbMsWLdiMd+9uhSUOaEay3QsV0zHhCxBBF4Yc6pK/Dx8y3wqLp4Sco90jsxzvYra6mLuXVIWdp7tUWPn6wGeeucAhjbTPYZ2aTsmpmNCFiFVNELSGMxN0nncT/+4h96gBk7x5sfHWXdPjetK2GOnz7Np1xEAnn73AEvnF4YmYn/92scEI4rfp03K5fgZR2mkju7nKgiZigi8kBQifd3X3B3b4Gtr80n6HH6/9oIne1TuzJf/0RJ3MB0oV7/1KSuvuzSq+QeYo/gwccfMzTtb+wlCJiMCLySFRNwkbWrKCsjxKmsEP5CHdz4kfB5F8UUTiJwzfeOjDmbk54Xy9R6guGACLSe7XT1nnK39aquLR+z3FYRURHLwQlJIxE3SpqrEz7oVi6mtLubGeTO5o8r0inE+JHqDmv3Hz0ada5dFhvL1OR5WLLmUnBhukjDQ2k9KJYVMR0bwQlJIxE0y8ngYMBnbsLONh2+ZT67PE5V6caKByXk+13v9ZFNDWKcnJ0GHn40gZCoygheShnM1aiI8v7ONnr6BtE6gu5c1d9dwV3VxaITu80Dh5Lyw85565wBA2L0C3b1xbYGdvjeCkKnICF5ICepbAjy749BA3bslwM6Hw4muHt78+DjHTveEndtv6JDFge0tP3/2lJg9W5WCVbeKq6SQ+YjACylBZCWNbRbmnGj1KIURw0XsD9taWb+9NbRyNdfn4af/YT57jpxiW/PJsPz9pdMnUV6Yn7TfRRBSBUnRCClBTVmBaRRmYVij8khv+FgukbYXjY3d2em5HYeiJmf3d5zhrifdG4ELQiYhAi+kBFUlflbdWoFd/GIbj/kn5JJr+ctoEm+3l+PzcKKrJ1R6GYn0ZhWyARF4IWWorS7mzkXFIXsBexS+5u4aFhRNSfg6l82YxLp7apienxf3uI6unrj7BSHdEYEXUor5s6fgiRjFNx3rovHo6ZjnRBqRHThxlqZjXcyfHf+h8NbHxyVNI2Q0IvBCylDfEmDVi3vDKl76+g3Wb28NTcAq4KpSP851TDk+D0sdPVuDhuahjQ1s+qANT+z1TiFrYkHIVETghZTBnlB1ooE9R06Fcu8aCHT3hTXU/ouiKdx73aWhxiH2cdsOBvB6VMiR0rm6VTH4CltBSHdE4IWkUt8S4Deb9yeUCnHaGzhH6JFNO/Z3nAmbbN1+MEDTsS7uvvaSqGuOy/Hwl3Nncn35DK6cMzUs/fMVy4lSEDIVqYMXksZQHCVhwN5gw842TnT18EZTB/2xvAYiWL+9lRvnF0Zt7zof5JV97a7nbNp1hEWXFIjpmJCxyAheSBpujpKJ8PzONl5rbEcbiRZFmmkc/4RccuOYjLmxfrs04BYyFxF4IWkMxVHSxvlQ0JqYk6QKmDp+4Ato0IDNTR2sW7GYpfNmxp1cdfJh2ynW1onIC5mJCLyQNJz9UgdLz9iEterL8bDii2XRbfcwc+id5/rDtr3xUQcAK6+7lDsXFbOo1O96buR1HtrUICIvZCRJy8ErpZ4GbgE6tNYVybqPkNpUlfiHNJFp92d9ec9RbqqYRW11Mc0nzsbMozsJGmaHp7c/OU5Pn4HXo7h3SRn1rQF2HAzEthzW8PALeygvzJdJVyGjSOYI/nfAV5J4fSEDsWvh391/glUv7qW+JWCWQCaYW3+9sT3kH99vaJ565wCfm5mPGuT0oMORUhAyhaQJvNZ6C/BZsq4vZCaxWv2tt3Lrg8m8oQkTc0ObHZ98gyTl7VWzsrJVyCTGPAevlFqhlNqhlNpx/PjxsQ5HGGPsHLwHUGqgQXZViZ8nv7WQ575/NYWDeMxce9k0fB6FR5m2wcsri7i+fMag9+6zOkkJQqYw5gKvtX5Ca71Qa71w+vTpYx2OMMbYOXiPx/R+t9M0zv1/c3Vp3Gu8++lJVt1awd9bk7sAb3wUO4fvtf4vkFG8kGmMucALmc9QVrOC2W7P0BpDQ09f9Ki6pqwgbr170NCs396Kf0IuW5tP8vzOtqjVsGE4Zl/Fn0bIJGQlq5BUhrqaFUwB93kUvUEdGlUvrywKnVdV4uenX63gwY0NMa+xu+0Uu9sa8Cgz/+71qtCqWI8izMsmqE1rBENLr1Yhs0jaCF4ptQ54HyhXSrUppb6XrHsJqctwVrNWlfi5Y+GcMF/4yFH83iOnErq/3Qnq6wvnUFtdzI3zZnLRpNzoA+2Z2cHKbQQhjUhmFc1dWutZWuscrXWR1vq3ybqXkLoMZzUrwLLKopD7o1tuPHETA/O+yyuLWF5ZxBtNHZzo6o06xjDMbwuSohEyCcnBC0llOKtZ7fPijeKXVxbhS+DTq4DvLC6lqsTP42996mpe5rEePpGVO4KQ7ojAC0mnqsTPD264bMirRCNH8eu3HwqzFPB4Bv/4amD1lmbu+8MHvN4Yu5Lm5orCmJU7gpCuiMALKYs9ircJGpqHX9hDfUuArc0n6euPVxoTzqZdR4jlPGxoeGH3kVDlzlCcLwUhlRGBF0aFoZZK2iyrLApbhWpbCtSUFSTsGJkIpnOlGvJcgSCkMiLwQtKxSyV/+UoT33xq65BEvqrEz6pbK0IdnjTwzI5DAPzl3JmxTwRKLpowpDgrZk8e8lyBIKQyIvBC0hlu4w+b2uriMDHvD5qukYOZkB0+dW5IIr+r7RRd5/rY2nxScvBCRiACLySdWP4yQ2F6hP/Ma43tNB3rYtVXK0KpmsgS9v6gpvWzbrwK8vO8gxqVgTkhO5xvGoKQiojAC0lnMH+ZRFhWWYTXkXTXGh7a2MD/2PxJaFWqdplE1ZgrVbt6gqHaeQWUFkxgbmG+671kolXIFETghVFhMH+Zwagq8fPIrRVhE6saaOs8P6TrKCAvx8MVc6bSeKwr5nFer0y0CumPeNEIo0Kkv8z67YeomD2F2urihK9hH/vjTQ0MoR83CjN94/Mo7lg4h/w8H6u3NMc9p3iIE7SCkIrICF4YFdxq2h/aOPReqLXVxfzstgXEa/DkbMYN5khfAdeXz2D+7Cn8+95jg95nf8cZ7npS8vBCeiMCL4wayyqLwoR5uA2va6uLeWbl1SydN9O1Fr7HZQFUUMMr+9p5cGMDB092J3QfycML6Y4IvDBqVJX4o2rXtTZTLkMVebvD07Mrr+aiieFVOef6jEErZhTE9ZQH8HnFOlhIb0TghVHFrXbd0IQsCIZKVYmff7ixPGp7ZIreOdJXQI5X0R8jka8cP4KQzojAC6NKWANth4LaFgTDoba6mCWXT4t7jFPLPQqmTx4Xc6JWWz9BQ0uKRkhrROCFUcdOr/zcMVl6of1Q/+7Ln0v42KCGw4Fzgx4n1sFCuiMCL4wZtdXF3LmoOG7npkQZ6ZG2R0G/ofnpn8Q6WEhfROCFMWWwzk2JYtfZJ8pgR9rpm95+g+eH+dARhLFGBF4YUxLpv5rode6+9pKEj585OW/wgyyG0h5QEFIJEXhhzBmpUXz++JyEjlPA8TM9CR2X61UsrywaciyCkAqIwAtjTuQofriNr2vKCgatbQe7Qmbw69VWF7NuxWLxhhfSFhF4ISVYVllEXs6FWQpXlfi5vnzGiMV0vKtHvOGFtEYEXkgJRsJSGGBafuK59cF4ZV+7eMMLaY0IvJAyBLp7CRqmpXBv3/DSNMsri0Z0BaqhzUoaWfAkpCMi8ELK4J+QG6pYMaz3wyGys9OF4lHiSSOkJyLwQsoQ6O4daL8H7DlyasjX2Np80rWz04VwyxdmyUSrkJaIwAspg3Ox0nDLJWvKCkIllyPF9oOfjej1BGG0EIEXUoaRKJeMbCwyEhzuPM99f/hgRK8pCKOBCLyQUoxEueSyyqKE6uGHwqZdR4bsWS8IY40IvJBSOMslg4bm4Rf2DKsZyEiP4gGefid+H1dBSDVE4IWUwy6X1JiOjsPp+BTZHnAk2H/8rIzihbRCBF5IOWrKCvA6nCGNYbT1qyrxU3HxlBGP7cGNDbLoSUgbROCFlKOqxM+qWyvC2uwZGh7a2MCK3+9IWGC/cVVxUuK764n3ReSFtEAEXkhJaquL+dltC8JEXmPaB9yx+r2ERvO11cXcdsXsEY+tN6hZ/th7UlkjpDwi8ELK4ibyYI3mE0zZ/PrOK5lbmJ+U+DbtOsLlD70keXkhZVF6pJf9XQALFy7UO3bsGOswhBRjbV0rD21siGq8oYCl82Zy73WXxl1pWt8SYPlj7yU1RoDSggn88utXyKpXYVRRStVrrRe67hOBF9KBtXWt/HhTQ6iVnhMFXDpjEt+95hJqq93z7o++1MjqLaNX5uj1QPnMfB65bYEIvpBUROCFjKC+JcDqtz7ltcb2mH4zcwvzqSzxs6yyKEpY7/vDB2zadWQUIo2NCL8w0ojACxnF2rpWfrKpgWCcj65HwUJLQD8720vZ9Ence92l/Nv7B8dc5IdCrleR6/WgPIq5hfncf9NceTAIYYyZwCulvgL8M+AFntJaPxrveBF4IVFCo/l97Qk3xVbAVaV+9nec4bPuvmSGJ8RBYT6AUTAhx4vHq+jr1+R6FT6fhzyfl4unjGPqhFym5+e5fhsTBhgTgVdKeYGPgaVAG7AduEtrvS/WOSLwwlCpbwnw6MuNbD8odelCeuMBmh/96yGfF0/gk1kmuQjYr7Vu1lr3An8Abk3i/YQspKrEz7Mrr2bD96+mtrqYRaX+qLJKQUgHDKDsgf89otf0jejVwrkYOOR43wZURx6klFoBrAAoLk7OykMh86kq8Ye+xte3BNiws40TXT10dvdyuPMcRzrPJ5zKEYSxwhjh6yVT4BNCa/0E8ASYKZoxDkfIAJxib+MUfZvO7l4+O9vLRRNzmeqwJe7s7qX5xFlOneujL95MriCMMCOdUkmmwB8GnJ6tRdY2QRh13EQ/ESIfDLEeCrH2dXb3svfIac72BlGYZZLK0TTWbjJu79OAYSDfNrKQ4ebg45FMgd8OXK6UugRT2O8EapN4P0EYcYb7YBCEVCBpAq+17ldK/T/AnzHLJJ/WWu9N1v0EQRCEcJKag9davwS8lMx7CIIgCO6Im6QgCEKGIgIvCIKQoYjAC4IgZCgi8IIgCBlKSrlJKqWOAy3DPH0acGIEw0km6RQrpFe86RQrpFe86RQrpFe8FxJridZ6utuOlBL4C0EptSOW4U6qkU6xQnrFm06xQnrFm06xQnrFm6xYJUUjCIKQoYjAC4IgZCiZJPBPjHUAQyCdYoX0ijedYoX0ijedYoX0ijcpsWZMDl4QBEEIJ5NG8IIgCIIDEXhBEIQMJe0FXin1FaVUk1Jqv1LqgTGM42mlVIdSao9j20VKqVeVUp9Y//Vb25VS6l+smD9USlU6zvm2dfwnSqlvJynWOUqpzUqpfUqpvUqpv0vVeJVS45RS25RSu61Y/7O1/RKlVJ0V03qlVK61Pc96v9/aX+q41o+s7U1Kqb8a6Vgj4vYqpT5QSr2YyvEqpQ4qpRqUUruUUjusbSn3OXDcZ6pS6jml1EdKqUal1OJUjFcpVW79Te2f00qp+0Y9Vq112v5g2hB/CpQBucBuYN4YxbIEqAT2OLb9E/CA9foB4B+t1zcDL2P2eagB6qztFwHN1n/91mt/EmKdBVRar/Mxm6PPS8V4rXtOsl7nAHVWDM8Ad1rbVwPft17/LbDaen0nsN56Pc/6fOQBl1ifG28SPw8/BNYCL1rvUzJe4CAwLWJbyn0OHLH9K3C39ToXmJrK8Vr38wLHgJLRjjUpv9Bo/QCLgT873v8I+NEYxlNKuMA3AbOs17OAJuv148BdkccBdwGPO7aHHZfEuF8AlqZ6vMAEYCdmb98TgC/yc4DZf2Cx9dpnHaciPxvO45IQZxHwOvAl4EXr/ikZL+4Cn5KfA2AKcACrOCTV43Vc/0bg3bGINd1TNG6NvS8eo1jcmKm1Pmq9PgbMtF7HinvUfx8rJXAl5sg4JeO10h27gA7gVczRbKfWut/lvqGYrP2ngILRitXi18B/ZKCHckEKx6uBV5RS9UqpFda2lPwcYH6TOQ78Tyv99ZRSamIKx2tzJ7DOej2qsaa7wKcN2nz8plRNqlJqErABuE9rfdq5L5Xi1VoHtdZXYI6MFwGfH9uIYqOUugXo0FrXj3UsCXKt1roSuAn4gVJqiXNnKn0OML/hVAKPaa2vBM5ipjlCpFi8WHMtXwWejdw3GrGmu8CnemPvdqXULADrvx3W9lhxj9rvo5TKwRT3NVrr51M9XgCtdSewGTPFMVUpZXckc943FJO1fwpwchRjvQb4qlLqIPAHzDTNP6dqvFrrw9Z/O4CNmA/QVP0ctAFtWus66/1zmIKfqvGC+eDcqbVut96PaqzpLvChxt7Wk/JO4I9jHJOTPwL2rPe3MXPd9vZvWTPnNcAp62vbn4EblVJ+a3b9RmvbiKKUUsBvgUat9a9SOV6l1HSl1FTr9XjMuYJGTKH/WoxY7d/ha8Ab1kjpj8CdVtXKJcDlwLaRjBVAa/0jrXWR1roU8/P4htb6m6kYr1JqolIq336N+e+3hxT8HABorY8Bh5RS5damvwT2pWq8FncxkJ6xYxq9WJM1sTBaP5izzx9j5mUfGsM41gFHgT7Mkcb3MHOprwOfAK8BF1nHKuA3VswNwELHdb4L7Ld+/u8kxXot5lfDD4Fd1s/NqRgv8AXgAyvWPcDD1vYyTMHbj/n1N8/aPs56v9/aX+a41kPW79AE3DQKn4nrGaiiSbl4rZh2Wz977f9/UvFz4LjPFcAO6/OwCbOyJCXjBSZifhub4tg2qrGKVYEgCEKGku4pGkEQBCEGIvCCIAgZigi8IAhChiICLwiCkKGIwAuCIGQoIvBCVqKUekiZ7pQfWm5/1Zbb34Sxjk0QRgopkxSyDqXUYuBXwPVa6x6l1DRMZ8L3MOuPT4xpgIIwQsgIXshGZgEntNY9AJagfw2YDWxWSm0GUErdqJR6Xym1Uyn1rOXdY3uo/5MyfdS3KaUus7bfoZTao0zv+i1j86sJwgAygheyDkuo38G0H34N04P9Lcs/ZqHW+oQ1qn8ecwXpWaXU/ZirT1dZxz2ptf65UupbwNe11rcopRqAr2itDyulpmrTO0cQxgwZwQtZh9b6DFAFrMC0n12vlPpOxGE1mE033rWsir+N2bDBZp3jv4ut1+8Cv1NK3YPZ5EEQxhTf4IcIQuahtQ4CbwJvWiPvb0ccooBXtdZ3xbpE5Gut9UqlVDXw10C9UqpKa31yZCMXhMSREbyQdSizX+bljk1XAC1AF2YLQ4CtwDWO/PpEpdTnHOd8w/Hf961jLtVa12mtH8b8ZuC0eRWEUUdG8EI2Mgn475YNcT+mS98KTGvXf1dKHdFa32ClbdYppfKs836M6VwK4FdKfQj0WOcB/FfrwaEwHQN3j8YvIwixkElWQRgizsnYsY5FEOIhKRpBEIQMRUbwgiAIGYqM4AVBEDIUEXhBEIQMRQReEAQhQxGBFwRByFBE4AVBEDKU/wOAnbvBbUa6TgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.plot(loss_monitor.loss, '.')\n",
    "plt.xlabel('Steps')\n",
    "plt.ylabel('Loss')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由上可知，通过量子模拟得到的量子版词嵌入模型也能很好的完成嵌入任务。当数据集大到经典计算机算力难以承受时，量子计算机将能够轻松处理这类问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 参考文献\n",
    "\n",
    "[1] Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean. [Efficient Estimation of Word Representations in\n",
    "Vector Space](https://arxiv.org/pdf/1301.3781.pdf)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.8.8 ('base')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "vscode": {
   "interpreter": {
    "hash": "d62cf896b9ca57de08105ce3983377439eacacf6f6599f9150bf400edf4fa4b8"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
